Skip to content

Commit

Permalink
cogmentlab install no longer requires root access (#10)
Browse files Browse the repository at this point in the history
# Description

- Installing cogment in a "known" directory defined by
`COGMENT_LAB_HOME` variable defaulting to `~/.cogment_lab`
- Installing cogment using a full python procedure (work also on
windows) by importing back the
[download_cogment.py](https://github.com/cogment/cogment-verse/blob/main/cogment_verse/utils/download_cogment.py)
script from Cogment Verse


## Type of change

Bug fix (non-breaking change which fixes an issue)

---------

Co-authored-by: ariel <[email protected]>
  • Loading branch information
cloderic and RedTachyon authored Feb 9, 2024
1 parent 972c1b9 commit 709a206
Show file tree
Hide file tree
Showing 7 changed files with 180 additions and 55 deletions.
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ __pycache__/
# Virtualenv
/env
/venv
/.venv

# Python egg metadata, regenerated from source files by setuptools.
/*.egg-info
Expand Down Expand Up @@ -42,4 +43,7 @@ tutorial/*.html
.idea
vizdoom.ini

# mypy
.mypy_cache

lib/
9 changes: 7 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,12 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.

## Unreleased

## v0.1.2 - 2024-02-02
### Fixed
- `cogmentlab install` no longer requires root access

### Changed
- The `cogmentlab` command now uses the `cogment` executable stored in `$COGMENT_LAB_HOME/cogment`. This might require rerunning `cogmentlab install` or updating the environment variable.


## v0.1.2 - 2024-02-02

Expand All @@ -34,8 +39,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
### Changed
- Dropped OpenCV as a requirement


## v0.1.0 - 2024-01-17

### Added

- Initial release
41 changes: 19 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,8 @@

# Human + AI = ❤️


## <a href="https://cogment.ai/cogment_lab"><strong>Docs</strong></a> | <a href="https://ai-r.com/blog"><strong>Blog</strong></a> | <a href="https://discord.gg/kh3t6esJRy"><strong> Discord </strong></a>


[![Package version](https://img.shields.io/pypi/v/cogment-lab?color=%23007ec6&label=pypi%20package)](https://pypi.org/project/cogment-lab)
[![Downloads](https://pepy.tech/badge/cogment-lab)](https://pepy.tech/project/cogment-lab)
[![Supported Python versions](https://img.shields.io/pypi/pyversions/cogment-lab.svg)](https://pypi.org/project/cogment-lab)
Expand All @@ -24,7 +22,7 @@ It's the perfect tool for when you want to interact with your environment yourse

1. Activate your venv, conda env, or whatever you use to keep your python environment clean.
2. Install cogment_lab with `pip install cogment_lab`
3. Install cogment with `cogmentlab install`
3. Install cogment in `COGMENT_LAB_HOME` folder with `cogmentlab install` (this environment variable defaults to `~/.cogment_lab`)
4. In a separate terminal, run `cogmentlab launch base` to start the orchestrator and datastore. Keep it open.
5. Run the tutorials, examples, or whatever you want to do.

Expand All @@ -48,39 +46,37 @@ allowing you to do your research without worries.

Cogment Lab is inherently asynchronous - but if you're not familiar with async python, don't worry about it.
The only things you need to remember are:

- Wrap your code in `async def main()`
- Run it with `asyncio.run(main())`
- When calling certain functions use the `await` keyword, e.g. `data = await cog.get_episode_data(...)`

If you are familiar with async programming, there's a lot of interesting things you can do with it - go crazy.


## Terminology

- A `service` is anything that interacts with the Cogment orchestrator. It can be an environment or an actor, including human actors.
- An `actor` in particular is the service that interacts with an environment, and often wraps an `agent`. The internal structure of an actor is entirely up to the user
- An `agent` is what we typically think of as an agent in RL - something that perceives its environment and acts upon it. We do not attempt to solve the agent foundation problem in this documentation.
- An `agent` is simultaneously the part of the environment that's taking an action - multiagent environments may have several agents, so we need to assign an actor to each agent.


## Known rough edges

- When running the web UI, you can open the tab only once per launched process. So if you open the UI, you can run however many trials you want, as long as you don't close it. If you do close it, you should kill the process and start a new one.


## Local installation

- Requires Python 3.10
- Install requirements in a virtual env with something similar to the following

```console
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install -r requirements.txt
$ pip install -e .
```
- For the examples you'll need to install the additional `examples_requirements.txt`.
```console
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install -r requirements.txt
$ pip install -e .
```

- For the examples you'll need to install the additional `examples_requirements.txt`.

### Apple silicon installation

Expand All @@ -101,6 +97,7 @@ Run `cogmentlab launch base`.
Then, run whatever scripts or notebooks.

Terminology:

- Model: a relatively raw PyTorch (or other?) model, inheriting from `nn.Module`
- Agent: a model wrapped in some utility class to interact with np arrays
- Actor: a cogment service that may involve models and/or actors
Expand All @@ -110,15 +107,15 @@ Terminology:
People having maintainers rights of the repository can follow these steps to release a version **MAJOR.MINOR.PATCH**. The versioning scheme follows [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

1. Run `./scripts/create_release_branch.sh MAJOR.MINOR.PATCH`, this will automatically:
- update the version of the package, in `cogment_lab/version.py`,
- create a release branch with the changes at `release/vMAJOR.MINOR.PATCH` and push it.
- update the version of the package, in `cogment_lab/version.py`,
- create a release branch with the changes at `release/vMAJOR.MINOR.PATCH` and push it.
2. On the release branch:
- Make sure the changelog, at `CHANGELOG.md`, reflects the changes since the last release,
- Fix any issue, making sure that the build passes on CI,
- Commit and push any changes.
- Make sure the changelog, at `CHANGELOG.md`, reflects the changes since the last release,
- Fix any issue, making sure that the build passes on CI,
- Commit and push any changes.
3. Run `./scripts/tag_release.sh MAJOR.MINOR.PATCH`, this will automatically:
- create the specific version section in the changelog and push it to the release branch,
- merge the release branch in `main`,
- create the release tag and,
- update the `develop` to match the latest release.
- create the specific version section in the changelog and push it to the release branch,
- merge the release branch in `main`,
- create the release tag and,
- update the `develop` to match the latest release.
4. The CI will automatically publish the package to PyPI.
38 changes: 9 additions & 29 deletions cogment_lab/cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,11 @@

import argparse
import logging
import os
import subprocess
import sys

from cogment_lab.cli.download_cogment import download_cogment
from cogment_lab.constants import COGMENT_LAB_HOME


TEAL = "\033[36m"
RESET = "\033[0m"
Expand All @@ -36,34 +37,13 @@
sys.path.insert(0, "..")


def install_cogment(path: str | None = None):
def install_cogment():
install_dir = COGMENT_LAB_HOME
try:
subprocess.run(
[
"curl",
"--silent",
"-L",
"https://raw.githubusercontent.com/cogment/cogment/main/install.sh",
"--output",
"install-cogment.sh",
],
check=True,
)
subprocess.run(["chmod", "+x", "install-cogment.sh"], check=True)
cmd = ["sudo", "./install-cogment.sh"]
if path:
cmd += ["--install-dir", path]
cmd += ["--version", "2.19.1"]
if os.getenv("GITHUB_ACTIONS") == "true":
cmd = cmd[1:] # Remove sudo for github actions
subprocess.run(cmd, check=True)
logging.info("Cogment installed successfully.")
except subprocess.CalledProcessError as e:
cogment_path = download_cogment(install_dir, "2.19.1")
logging.info(f"Cogment installed successfully in [{cogment_path}].")
except Exception as e:
logging.error(f"Installation failed: {e}")
finally:
if os.path.exists("install-cogment.sh"):
os.remove("install-cogment.sh")
logging.info("Cleanup completed.")


def main():
Expand All @@ -87,7 +67,7 @@ def main():
args = parser.parse_args()

if args.command == "install":
install_cogment(args.path)
install_cogment()
elif args.command == "launch":
from cogment_lab.cli import launch

Expand Down
131 changes: 131 additions & 0 deletions cogment_lab/cli/download_cogment.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Copyright 2024 AI Redefined Inc. <[email protected]>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from __future__ import annotations

import json
import os
import platform
import re
import stat
from enum import Enum
from tempfile import mkdtemp
from urllib.request import urlopen, urlretrieve


class Arch(Enum):
AMD64 = "amd64"
ARM64 = "arm64"


def get_current_arch() -> Arch:
py_machine = platform.machine()
if py_machine in ["x86_64", "i686", "AMD64"]:
return Arch.AMD64

if py_machine in ["arm64"]:
return Arch.ARM64

raise RuntimeError(f"Unsupported architecture [{py_machine}]")


class Os(Enum):
WINDOWS = "windows"
LINUX = "linux"
MACOS = "macos"


def get_current_os() -> Os:
py_system = platform.system()
if py_system == "Darwin":
return Os.MACOS
if py_system == "Windows":
return Os.WINDOWS
if py_system == "Linux":
return Os.LINUX

raise RuntimeError(f"Unsupported os [{py_system}]")


def get_latest_release_version() -> str:
res = urlopen("https://api.github.com/repos/cogment/cogment/releases/latest")

parsedBody = json.load(res)

return parsedBody["tag_name"]


def download_cogment(
output_dir: str | None = None,
desired_version: str | None = None,
desired_arch: Arch | None = None,
desired_os: Os | None = None,
):
"""
Download a version of cogment
Parameters:
- output_dir (string, optional): the output directory, if undefined a temporary directory will be used.
- desired_version (string, optional): the desired version,
if undefined the latest released version (excluding prereleases) will be used.
- desired_arch (Arch, optional): the desired architecture,
if undefined the current architecture will be detected and used.
- os (Os, optional): the desired os, if undefined the current os will be detected and used.
Returns:
path to the downloaded cogment
"""
if not output_dir:
output_dir = mkdtemp()
else:
output_dir = os.path.abspath(output_dir)
os.makedirs(output_dir, exist_ok=True)

if not desired_version:
desired_version = get_latest_release_version()

try:
desired_version = re.findall(r"[0-9]+.[0-9]+.[0-9]+(?:-[a-zA-Z0-9]+)?", desired_version)[0]
except RuntimeError as exc:
raise RuntimeError(f"Desired cogment version [{desired_version}] doesn't follow the expected patterns") from exc

if desired_arch is None:
desired_arch = get_current_arch()

if desired_os is None:
desired_os = get_current_os()

cogment_url = (
"https://github.com/cogment/cogment/releases/download/"
+ f"v{desired_version}/cogment-{desired_os.value}-{desired_arch.value}"
)

cogment_filename = os.path.join(output_dir, "cogment")
if desired_os == Os.WINDOWS:
cogment_url += ".exe"
cogment_filename += ".exe"

try:
cogment_filename, _ = urlretrieve(cogment_url, cogment_filename)
except Exception as exc:
raise RuntimeError(
f"Unable to retrieve cogment version [{desired_version}] for arch "
+ f"[{desired_arch}] and os [{desired_os}] from [{cogment_url}] to [{cogment_filename}]"
) from exc

# Make sure it is executable
cogment_stat = os.stat(cogment_filename)
os.chmod(cogment_filename, cogment_stat.st_mode | stat.S_IEXEC)

return cogment_filename
6 changes: 4 additions & 2 deletions cogment_lab/cli/launch.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,13 @@
import logging
import subprocess

from cogment_lab.constants import COGMENT_LAB_HOME


def launch_service(service_name: str):
cogment_path = COGMENT_LAB_HOME / "cogment"
try:
process = subprocess.Popen(["cogment", "services", service_name])
process = subprocess.Popen([cogment_path, "services", service_name])
logging.info(f"{service_name} launched successfully. PID: {process.pid}")
return process
except Exception as e:
Expand Down Expand Up @@ -52,7 +55,6 @@ def launch_main(command: str):
if process:
processes.append(process)

# Optional: Wait for all subprocesses to complete
for process in processes:
process.wait()

Expand Down
6 changes: 6 additions & 0 deletions cogment_lab/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,10 @@
# See the License for the specific language governing permissions and
# limitations under the License.

import os
from pathlib import Path


DEFAULT_RENDERED_WIDTH = 1024

COGMENT_LAB_HOME = os.getenv("COGMENT_LAB_HOME", Path.home() / ".cogment_lab")

0 comments on commit 709a206

Please sign in to comment.