Skip to content

Commit

Permalink
cogmentlab install no longer requires root access
Browse files Browse the repository at this point in the history
  • Loading branch information
cloderic committed Jan 24, 2024
1 parent 993f8a5 commit 5a03a87
Show file tree
Hide file tree
Showing 7 changed files with 178 additions and 51 deletions.
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ __pycache__/
# Virtualenv
/env
/venv
/.venv

# Python egg metadata, regenerated from source files by setuptools.
/*.egg-info
Expand Down Expand Up @@ -42,4 +43,7 @@ tutorial/*.html
.idea
vizdoom.ini

# mypy
.mypy_cache

lib/
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,27 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.

## Unreleased

## v0.1.1 - 2024-01-22
### Fixed

- `cogmentlab install` no longer requires root access

## v0.1.1 - 2024-01-22

### Added

- Added guided tutorial notebooks
- Added an option to customize the orchestrator and datastore ports
- Added ParallelEnvironment as a default export from envs
- Added a placeholder image for the web UI

### Fixed

- Updated the uvicorn dependency to require the [standard] option
- Fixed a breaking bug in ParallelEnv
- Fixed some type issues, ignore some spurious warnings

### Changed

- Dropped OpenCV as a requirement

## v0.1.0 - 2024-01-17
Expand Down
41 changes: 19 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,8 @@

# Human + AI = ❤️


## <a href="https://cogment.ai/cogment_lab"><strong>Docs</strong></a> | <a href="https://ai-r.com/blog"><strong>Blog</strong></a> | <a href="https://discord.gg/kh3t6esJRy"><strong> Discord </strong></a>


[![Package version](https://img.shields.io/pypi/v/cogment-lab?color=%23007ec6&label=pypi%20package)](https://pypi.org/project/cogment-lab)
[![Downloads](https://pepy.tech/badge/cogment-lab)](https://pepy.tech/project/cogment-lab)
[![Supported Python versions](https://img.shields.io/pypi/pyversions/cogment-lab.svg)](https://pypi.org/project/cogment-lab)
Expand All @@ -24,7 +22,7 @@ It's the perfect tool for when you want to interact with your environment yourse

1. Activate your venv, conda env, or whatever you use to keep your python environment clean.
2. Install cogment_lab with `pip install cogment_lab`
3. Install cogment with `cogmentlab install`
3. Install cogment in `COGMENT_LAB_HOME` folder with `cogmentlab install` (this environment variable defaults to `~/.cogment_lab`)
4. In a separate terminal, run `cogmentlab launch base` to start the orchestrator and datastore. Keep it open.
5. Run the tutorials, examples, or whatever you want to do.

Expand All @@ -48,39 +46,37 @@ allowing you to do your research without worries.

Cogment Lab is inherently asynchronous - but if you're not familiar with async python, don't worry about it.
The only things you need to remember are:

- Wrap your code in `async def main()`
- Run it with `asyncio.run(main())`
- When calling certain functions use the `await` keyword, e.g. `data = await cog.get_episode_data(...)`

If you are familiar with async programming, there's a lot of interesting things you can do with it - go crazy.


## Terminology

- A `service` is anything that interacts with the Cogment orchestrator. It can be an environment or an actor, including human actors.
- An `actor` in particular is the service that interacts with an environment, and often wraps an `agent`. The internal structure of an actor is entirely up to the user
- An `agent` is what we typically think of as an agent in RL - something that perceives its environment and acts upon it. We do not attempt to solve the agent foundation problem in this documentation.
- An `agent` is simultaneously the part of the environment that's taking an action - multiagent environments may have several agents, so we need to assign an actor to each agent.


## Known rough edges

- When running the web UI, you can open the tab only once per launched process. So if you open the UI, you can run however many trials you want, as long as you don't close it. If you do close it, you should kill the process and start a new one.


## Local installation

- Requires Python 3.10
- Install requirements in a virtual env with something similar to the following

```console
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install -r requirements.txt
$ pip install -e .
```
- For the examples you'll need to install the additional `examples_requirements.txt`.
```console
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install -r requirements.txt
$ pip install -e .
```

- For the examples you'll need to install the additional `examples_requirements.txt`.

### Apple silicon installation

Expand All @@ -101,6 +97,7 @@ Run `cogmentlab launch base`.
Then, run whatever scripts or notebooks.

Terminology:

- Model: a relatively raw PyTorch (or other?) model, inheriting from `nn.Module`
- Agent: a model wrapped in some utility class to interact with np arrays
- Actor: a cogment service that may involve models and/or actors
Expand All @@ -110,15 +107,15 @@ Terminology:
People having maintainers rights of the repository can follow these steps to release a version **MAJOR.MINOR.PATCH**. The versioning scheme follows [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

1. Run `./scripts/create_release_branch.sh MAJOR.MINOR.PATCH`, this will automatically:
- update the version of the package, in `cogment_lab/version.py`,
- create a release branch with the changes at `release/vMAJOR.MINOR.PATCH` and push it.
- update the version of the package, in `cogment_lab/version.py`,
- create a release branch with the changes at `release/vMAJOR.MINOR.PATCH` and push it.
2. On the release branch:
- Make sure the changelog, at `CHANGELOG.md`, reflects the changes since the last release,
- Fix any issue, making sure that the build passes on CI,
- Commit and push any changes.
- Make sure the changelog, at `CHANGELOG.md`, reflects the changes since the last release,
- Fix any issue, making sure that the build passes on CI,
- Commit and push any changes.
3. Run `./scripts/tag_release.sh MAJOR.MINOR.PATCH`, this will automatically:
- create the specific version section in the changelog and push it to the release branch,
- merge the release branch in `main`,
- create the release tag and,
- update the `develop` to match the latest release.
- create the specific version section in the changelog and push it to the release branch,
- merge the release branch in `main`,
- create the release tag and,
- update the `develop` to match the latest release.
4. The CI will automatically publish the package to PyPI.
36 changes: 9 additions & 27 deletions cogment_lab/cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,10 @@
import os
import subprocess
import sys
import tempfile

from cogment_lab.constants import COGMENT_LAB_HOME
from cogment_lab.cli.download_cogment import download_cogment

TEAL = "\033[36m"
RESET = "\033[0m"
Expand All @@ -36,34 +39,13 @@
sys.path.insert(0, "..")


def install_cogment(path: str | None = None):
def install_cogment():
install_dir = COGMENT_LAB_HOME / "bin"
try:
subprocess.run(
[
"curl",
"--silent",
"-L",
"https://raw.githubusercontent.com/cogment/cogment/main/install.sh",
"--output",
"install-cogment.sh",
],
check=True,
)
subprocess.run(["chmod", "+x", "install-cogment.sh"], check=True)
cmd = ["sudo", "./install-cogment.sh"]
if path:
cmd += ["--install-dir", path]
cmd += ["--version", "2.19.1"]
if os.getenv("GITHUB_ACTIONS") == "true":
cmd = cmd[1:] # Remove sudo for github actions
subprocess.run(cmd, check=True)
logging.info("Cogment installed successfully.")
except subprocess.CalledProcessError as e:
cogment_path = download_cogment(install_dir, "2.19.1")
logging.info(f"Cogment installed successfully in [{cogment_path}].")
except Exception as e:
logging.error(f"Installation failed: {e}")
finally:
if os.path.exists("install-cogment.sh"):
os.remove("install-cogment.sh")
logging.info("Cleanup completed.")


def main():
Expand All @@ -87,7 +69,7 @@ def main():
args = parser.parse_args()

if args.command == "install":
install_cogment(args.path)
install_cogment()
elif args.command == "launch":
from cogment_lab.cli import launch

Expand Down
133 changes: 133 additions & 0 deletions cogment_lab/cli/download_cogment.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# Copyright 2023 AI Redefined Inc. <[email protected]>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from enum import Enum
from tempfile import mkdtemp
import json
import logging
import os
import platform
import re
import shutil
import stat
import subprocess

from urllib.request import urlretrieve, urlopen


class Arch(Enum):
AMD64 = "amd64"
ARM64 = "arm64"


def get_current_arch():
py_machine = platform.machine()
if py_machine in ["x86_64", "i686", "AMD64"]:
return Arch.AMD64

if py_machine in ["arm64"]:
return Arch.ARM64

raise RuntimeError(f"Unsupported architecture [{py_machine}]")


class Os(Enum):
WINDOWS = "windows"
LINUX = "linux"
MACOS = "macos"


def get_current_os():
py_system = platform.system()
if py_system in ["Darwin"]:
return Os.MACOS
if py_system in ["Windows"]:
return Os.WINDOWS
if py_system in ["Linux"]:
return Os.LINUX

raise RuntimeError(f"Unsupported os [{py_system}]")


def get_latest_release_version():
res = urlopen("https://api.github.com/repos/cogment/cogment/releases/latest")

parsedBody = json.load(res)

return parsedBody["tag_name"]


def download_cogment(
output_dir=None,
desired_version=None,
desired_arch=None,
desired_os=None,
):
"""
Download a version of cogment
Parameters:
- output_dir (string, optional): the output directory, if undefined a temporary directory will be used.
- desired_version (string, optional): the desired version,
if undefined the latest released version (excluding prereleases) will be used.
- desired_arch (Arch, optional): the desired architecture,
if undefined the current architecture will be detected and used.
- os (Os, optional): the desired os, if undefined the current os will be detected and used.
Returns:
path to the downloaded cogment
"""
if not output_dir:
output_dir = mkdtemp()
else:
output_dir = os.path.abspath(output_dir)
os.makedirs(output_dir, exist_ok=True)

if not desired_version:
desired_version = get_latest_release_version()

try:
desired_version = re.findall(r"[0-9]+.[0-9]+.[0-9]+(?:-[a-zA-Z0-9]+)?", desired_version)[0]
except RuntimeError as e:
raise RuntimeError(f"Desired cogment version [{desired_version}] doesn't follow the expected patterns")

if not desired_arch:
desired_arch = get_current_arch()

if not desired_os:
desired_os = get_current_os()

cogment_url = (
"https://github.com/cogment/cogment/releases/download/" +
f"v{desired_version}/cogment-{desired_os.value}-{desired_arch.value}"
)

cogment_filename = os.path.join(output_dir, "cogment")
if desired_os == Os.WINDOWS:
cogment_url += ".exe"
cogment_filename += ".exe"

try:
cogment_filename, _ = urlretrieve(cogment_url, cogment_filename)
except Exception as e:
raise RuntimeError(
f"Unable to retrieve cogment version [{desired_version}] for arch " +
f"[{desired_arch}] and os [{desired_os}] from [{cogment_url}] to [{cogment_filename}]: [{e}]"
)

# Make sure it is executable
cogment_stat = os.stat(cogment_filename)
os.chmod(cogment_filename, cogment_stat.st_mode | stat.S_IEXEC)

return cogment_filename
4 changes: 3 additions & 1 deletion cogment_lab/cli/launch.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@
import logging
import subprocess

from cogment_lab.constants import COGMENT_LAB_HOME

def launch_service(service_name: str):
cogment_path = COGMENT_LAB_HOME / "bin/cogment"
try:
process = subprocess.Popen(["cogment", "services", service_name])
process = subprocess.Popen([cogment_path, "services", service_name])
logging.info(f"{service_name} launched successfully. PID: {process.pid}")
return process
except Exception as e:
Expand Down
5 changes: 5 additions & 0 deletions cogment_lab/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.

import os
from pathlib import Path

DEFAULT_RENDERED_WIDTH = 1024

COGMENT_LAB_HOME = os.getenv("COGMENT_LAB_HOME", Path.home() / ".cogment_lab")

0 comments on commit 5a03a87

Please sign in to comment.