Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install Python using uv. #248

Merged
merged 29 commits into from
Nov 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
7abc694
Use uv to install python.
robinjhuang Oct 28, 2024
567bef2
Use resource path for mac.
robinjhuang Nov 5, 2024
908797d
download UV on github action and build time pick platform
KenCorma Nov 6, 2024
1e77024
async command
KenCorma Nov 6, 2024
c9720b5
Delete some items on windows
KenCorma Nov 6, 2024
add68b4
cleaning up old code
KenCorma Nov 7, 2024
0dda8d3
Local UV and CI uses UV ver in Package.json
KenCorma Nov 8, 2024
f0bb02c
Removed some items from virtualEnviroment
KenCorma Nov 9, 2024
7147ea7
Remove Python stuff from CI build
KenCorma Nov 10, 2024
98c3669
Filter via string endsWith
KenCorma Nov 10, 2024
304217e
Small refactors.
robinjhuang Nov 12, 2024
f1b3f73
Add uv instructions to README.
robinjhuang Nov 12, 2024
9a04643
Empty space.
robinjhuang Nov 12, 2024
045d9f3
revert.
robinjhuang Nov 12, 2024
f36708b
Add execution permissions.
robinjhuang Nov 12, 2024
b0732a0
add fallback to installing requirements.
robinjhuang Nov 13, 2024
efc85a1
Download uv 0.5.1
robinjhuang Nov 13, 2024
84a2752
Remove unused.
robinjhuang Nov 14, 2024
2b54ee0
Fix.
robinjhuang Nov 14, 2024
1a07084
Update.
robinjhuang Nov 14, 2024
37a4c41
Increment frontend version.
robinjhuang Nov 14, 2024
7fcc040
Fix.
robinjhuang Nov 14, 2024
d062852
Switch order of download uv command.
robinjhuang Nov 14, 2024
3e8f481
Fix.
robinjhuang Nov 14, 2024
226c339
Fix commands.
robinjhuang Nov 14, 2024
4eb5adf
Not needed anymore.
robinjhuang Nov 14, 2024
40fbd25
Clean up.
robinjhuang Nov 14, 2024
19fbe61
Send logs for setting up python as well.
robinjhuang Nov 14, 2024
d78a3c3
Merge branch 'main' into rh-uv3
robinjhuang Nov 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/actions/build/windows/todesktop/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ runs:
shell: cmd
- run: yarn set version --yarn-path self
shell: cmd
- run: yarn run download:uv all
shell: powershell
- name: Make app
shell: powershell
env:
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -121,3 +121,6 @@ test-results

# Python venv
venv/

# UV
assets/uv
14 changes: 6 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,12 +85,6 @@ This will install a usable `yarn` binary. Then, in the root directory of this re
yarn install
```

Start the development server:

```bash
yarn start
```

## Setup Python

Make sure you have python 3.12+ installed. It is recommended to setup a virtual environment to run the python code.
Expand All @@ -117,7 +111,7 @@ With the python environment activated, install comfy-cli:
pip install comfy-cli
```

## Building/running
## Building/Running

First, initialize the application resources by running `make:assets:<gpu>`:

Expand All @@ -126,7 +120,11 @@ First, initialize the application resources by running `make:assets:<gpu>`:
yarn make:assets:[amd|cpu|nvidia|macos]
```

This command will install ComfyUI under `assets`, as well ComfyUI-Manager, and the frontend [extension](https://github.com/Comfy-Org/DesktopSettingsExtension) responsible for electron settings menu.
This command will install ComfyUI under `assets`, as well ComfyUI-Manager, and the frontend [extension](https://github.com/Comfy-Org/DesktopSettingsExtension) responsible for electron settings menu. The exact versions of each package is defined in `package.json`.

Second, you need to install `uv`. This will be bundled with the distributable, but we also need it locally.

`yarn download:uv`

You can then run `start` to build/launch the code and a live buildserver that will automatically rebuild the code on any changes:

Expand Down
3 changes: 2 additions & 1 deletion builder-debug.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ const debugConfig: Configuration = {
files: ['node_modules', 'package.json', '.vite/**'],
extraResources: [
{ from: './assets/ComfyUI', to: 'ComfyUI' },
{ from: './assets/python.tgz', to: 'python.tgz' },
{ from: './assets/uv/uv', to: 'uv/uv' },
{ from: './assets/uv/uvx', to: 'uv/uvx' },
{ from: './assets/UI', to: 'UI' },
],
beforeBuild: './scripts/preMake.js',
Expand Down
7 changes: 5 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,26 @@
"productName": "ComfyUI",
"repository": "github:comfy-org/electron",
"copyright": "Copyright © 2024 Comfy Org",
"version": "0.2.17",
"version": "0.3.0",
"homepage": "https://comfy.org",
"description": "The best modular GUI to run AI diffusion models.",
"main": ".vite/build/main.js",
"packageManager": "[email protected]",
"config": {
"frontendVersion": "1.3.43",
"comfyVersion": "0.2.7",
"managerCommit": "e629215c100c89a9a9d33fc03be3248069ff67ef"
"managerCommit": "e629215c100c89a9a9d33fc03be3248069ff67ef",
"uvVersion": "0.5.1"
},
"scripts": {
"clean": "rimraf .vite dist out",
"clean:uv": "rimraf assets/uv",
"clean:assets": "rimraf assets/.env assets/ComfyUI assets/python.tgz & yarn run clean:assets:dev",
"clean:assets:dev": "yarn run clean:assets:git && rimraf assets/python assets/override.txt & rimraf assets/cpython*.tar.gz & rimraf assets/requirements.*",
"clean:assets:git": "rimraf assets/ComfyUI/.git assets/ComfyUI/custom_nodes/ComfyUI_Manager/.git assets/ComfyUI/custom_nodes/DesktopSettingsExtension/.git",
"clean:slate": "yarn run clean & yarn run clean:assets & rimraf node_modules",
"clone-settings-extension": "git clone https://github.com/Comfy-Org/DesktopSettingsExtension.git assets/ComfyUI/custom_nodes/DesktopSettingsExtension",
"download:uv": "node scripts/downloadUV.js",
"download-frontend": "node scripts/downloadFrontend.js",
"make:frontend": "yarn run download-frontend && yarn run clone-settings-extension",
"format": "prettier --check .",
Expand Down
10 changes: 0 additions & 10 deletions scripts/checkAssetsMacos.sh

This file was deleted.

83 changes: 83 additions & 0 deletions scripts/downloadUV.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
const path = require("path");
const os = require('os');
const fs = require('fs-extra');
const axios = require('axios');
const tar = require('tar');
const extractZip = require('extract-zip');
const uvVer = require('../package.json').config.uvVersion;

const options = {
win32: {
zipFile: 'uv-x86_64-pc-windows-msvc.zip',
uvOutputFolderName: 'win',
zip: true,
},
darwin: {
zipFile: 'uv-aarch64-apple-darwin.tar.gz',
uvOutputFolderName: 'macos',
zip: false,
},
linux: {
zipFile: 'uv-x86_64-unknown-linux-gnu.tar.gz',
uvOutputFolderName: 'linux',
zip: false,
}
}

async function downloadUV() {

const allFlag = process.argv[2];
const baseDownloadURL = `https://github.com/astral-sh/uv/releases/download/${uvVer}/`;
if (allFlag)
{
if (allFlag === 'all') {
await downloadAndExtract(baseDownloadURL, options.win32);
await downloadAndExtract(baseDownloadURL, options.darwin);
await downloadAndExtract(baseDownloadURL, options.linux);
return;
}
if (allFlag === 'none') {
return;
}
}

const uvDownloaded = fs.existsSync(path.join('./assets', 'uv'));
if (!uvDownloaded) {
await downloadAndExtract(baseDownloadURL, options[os.platform()]);
return;
}
console.log('< UV Folder Exists, Skipping >');

};

async function downloadAndExtract(baseURL, options) {
const {
zipFile,
uvOutputFolderName,
zip
} = options;
const zipFilePath = path.join('./assets', zipFile);
const outputUVFolder = path.join('./assets', 'uv', uvOutputFolderName);
await fs.mkdir(outputUVFolder, {
recursive: true
});
const downloadedFile = await axios({
method: 'GET',
url: baseURL + zipFile,
responseType: 'arraybuffer'
});
fs.writeFileSync(zipFilePath, downloadedFile.data);
zip ? await extractZip(zipFilePath, {
dir: path.resolve(outputUVFolder)
}) : tar.extract({
sync: true,
file: zipFilePath,
C: outputUVFolder,
"strip-components": 1
});
await fs.unlink(zipFilePath);
console.log(`FINISHED DOWNLOAD AND EXTRACT UV ${uvOutputFolderName}`);
}

//** Download and Extract UV. Default uses OS.Platfrom. Add 'all' will download all. Add 'none' will skip */
downloadUV();
7 changes: 2 additions & 5 deletions scripts/makeComfy.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,8 @@ function makeAssets(gpuFlag) {
'yarn run make:frontend'
].join(' ');

if (gpuFlag === '--m-series') {
execSync(`${baseCommand} && ../scripts/checkAssetsMacos.sh python`, { stdio: 'inherit' });
} else {
execSync(baseCommand, { stdio: 'inherit' });
}

execSync(baseCommand, { stdio: 'inherit' });

// Rename custom_nodes/ComfyUI-Manager to manager-core
if (!fs.existsSync('assets/ComfyUI/custom_nodes/ComfyUI-Manager')) {
Expand Down
25 changes: 21 additions & 4 deletions scripts/todesktop/afterPack.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
const os = require('os');
const fs = require('fs/promises');
const path = require('path');
const { spawnSync } = require('child_process');

module.exports = async ({ appOutDir, packager, outDir }) => {
/**
Expand All @@ -13,7 +14,7 @@ module.exports = async ({ appOutDir, packager, outDir }) => {
* arch - number - the architecture of the app. ia32 = 0, x64 = 1, armv7l = 2, arm64 = 3, universal = 4.
*/

// The purpose of this script is to move the built python and comfy files from assets to the resource folder of the app
// The purpose of this script is to move comfy files from assets to the resource folder of the app
// We can not add them to extraFiles as that is done prior to building, where we need to move them AFTER

if (os.platform() === "darwin") {
Expand All @@ -22,8 +23,18 @@ module.exports = async ({ appOutDir, packager, outDir }) => {
const mainPath = path.dirname(outDir);
const assetPath = path.join(mainPath, 'app-wrapper', 'app', 'assets');
const resourcePath = path.join(appPath, "Contents", "Resources");
const result = await fs.rm(path.join(assetPath, "ComfyUI", ".git"), { recursive: true, force: true });
const result2 = await fs.cp(assetPath, resourcePath, { recursive: true });
// Remove these Git folders that mac's codesign is choking on. Need a more recursive way to just find all folders with '.git' and delete
await fs.rm(path.join(assetPath, "ComfyUI", ".git"), { recursive: true, force: true });
await fs.rm(path.join(assetPath, "ComfyUI", 'custom_nodes', 'manager-core', ".git"), { recursive: true, force: true });
await fs.rm(path.join(assetPath, "ComfyUI", 'custom_nodes', 'DesktopSettingsExtension', ".git"), { recursive: true, force: true });
// Move rest of items to the resource folder
await fs.cp(assetPath, resourcePath, { recursive: true });
// Remove other OS's UV
await fs.rm(path.join(resourcePath, 'uv', 'win'), { recursive: true, force: true });
await fs.rm(path.join(resourcePath, 'uv', 'linux'), { recursive: true, force: true });
await fs.chmod(path.join(resourcePath, 'uv', 'macos', 'uv'), '755');
await fs.chmod(path.join(resourcePath, 'uv', 'macos', 'uvx'), '755');

}

if (os.platform() === 'win32') {
Expand All @@ -32,6 +43,12 @@ module.exports = async ({ appOutDir, packager, outDir }) => {
const mainPath = path.dirname(outDir);
const assetPath = path.join(mainPath, 'app-wrapper', 'app', 'assets');
const resourcePath = path.join(path.dirname(appPath), "resources");
// Move rest of items to the resource folder
await fs.cp(assetPath, resourcePath, { recursive: true });
// Remove other OS's UV
await fs.rm(path.join(resourcePath, 'uv', 'macos'), { recursive: true, force: true });
await fs.rm(path.join(resourcePath, 'uv', 'linux'), { recursive: true, force: true });
}
}

//TODO: Linux
}
56 changes: 35 additions & 21 deletions scripts/todesktop/postInstall.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,43 +2,57 @@ const { spawnSync } = require("child_process");
const path = require("path");
const os = require('os');
const process = require("process");
//const fs = require('fs-extra');
const fs = require('fs-extra');

async function postInstall() {
const firstInstallOnToDesktopServers =
process.env.TODESKTOP_CI && process.env.TODESKTOP_INITIAL_INSTALL_PHASE;

if (!firstInstallOnToDesktopServers) return;

console.log('After Yarn Install' , os.platform());
console.log('After Yarn Install ' , os.platform());

if (os.platform() === "win32")
{
// Change stdio to get back the logs if there are issues.
const resultUpgradePip = spawnSync(`py`, ['-3.12', '-m', 'pip' ,'install' ,'--upgrade pip'],{shell:true,stdio: 'ignore'}).toString();
const resultInstallComfyCLI = spawnSync(`py`, ['-3.12 ','-m' ,'pip' ,'install comfy-cli'], {shell:true,stdio: 'ignore'}).toString();
console.log("Finish PIP & ComfyCLI Install");
const resultComfyManagerInstall = spawnSync('set PATH=C:\\hostedtoolcache\\windows\\Python\\3.12.7\\x64\\Scripts;%PATH% && yarn run make:assets:nvidia' ,[''],{shell:true,stdio: 'inherit'}).toString();
console.log("Finish Comfy Manager Install and Rehydration");
}

if (os.platform() === "darwin") {

const resultUpgradePip = spawnSync(`py`, ['-3.12', '-m', 'pip' ,'install' ,'--upgrade pip'],{shell:true,stdio: 'ignore'}).toString();
const resultInstallComfyCLI = spawnSync(`py`, ['-3.12 ','-m' ,'pip' ,'install comfy-cli'], {shell:true,stdio: 'ignore'}).toString();
const resultComfyManagerInstall = spawnSync('yarn run make:assets:macos' ,[''],{shell:true,stdio: 'inherit'}).toString();

// Do not delete, useful if there are build issues with mac
// TODO: Consider making a global build log as ToDesktop logs can be hit or miss
/*
fs.createFileSync('./src/macpip.txt');
fs.writeFileSync('./src/macpip.txt',JSON.stringify({
log: result.stdout.toString(),
err:result.stderr.toString()
}));
*/
console.log("Finish Python & Comfy Install for Mac");
if (os.platform() == 'darwin')
{
// Python install pip and install comfy-cli
const resultUpgradePip = spawnSync(`python3.12`, ['-m', 'pip', 'install', '--upgrade pip'], {
shell: true,
stdio: 'ignore',
encoding: 'utf-8',
});
const resultInstallComfyCLI = spawnSync(`python3.12`, ['-m', 'pip', 'install comfy-cli'], {
shell: true,
stdio: 'inherit',
encoding: 'utf-8',
});
// Finally add this python to path and then run the Assets Make for MacOS
const resultComfyManagerInstall = spawnSync('export PATH="/Library/Frameworks/Python.framework/Versions/3.12/bin:$PATH" && yarn run make:assets:macos', [''], {
shell: true,
stdio: 'inherit',
encoding: 'utf-8',
});

}

//TODO: Linux

// Remove python stuff
await fs.rm(path.join('./assets', 'python'), { recursive: true, force: true });
await fs.rm(path.join('./assets', 'python.tgz'), { force: true });
fs.readdirSync(path.join('./assets')).forEach((tgzFile) => {
if (tgzFile.endsWith('.gz')) {
fs.rmSync(path.join('./assets', tgzFile));
}
});

};

postInstall();
postInstall();
Loading
Loading