Skip to content

Commit

Permalink
Update the reminder plugin and add a test for it (#10)
Browse files Browse the repository at this point in the history
* Fix metadata.yaml to work with current ops

* Fix integration tests

* Add github workflows and issue templates

* Replace apt dist-upgrade by apt upgrade

* Lint metadata.yaml

* Shorten summary of the charm

* Largely ignore inclusive naming issues for now

* Add our standard tox/pyproject structure w/o enabling everything

* Don't use the full test suite of operator-workflows

* Don't hide files from inclusive names test

* Only do unit tests in unit tests !

* Expand summary a bit

* Removing duplicate apt-get update/upgrade

* Change licence into license

* Create environment module

* Render fn outputs immutable in environment.py

* Fix incorrect comment

* Revert config naming back to "licence"

* Add some context to a comment with link to ref

* Fix tests still using "license" as config name

* Simplify `enable_smtp_auth` definition

* Add fixtures in conftest.py for integration tests

* Change env functions to be generators

* Transform missing_config_settings into generator

* Use typing.Tuple in environment.py

* Add/Fix copyright notices

* Add s3_tls config option

* Add s3 integration test

* Fix typo

* Add local_mode buildarg to the Dockerfile

* Rework integ tests to make s3 tests work

* Adapt integ tests to use the build image job of operator-workflows

* Relaunch CI

* Test without build-args

* Fix input name

* Change image-build-args into list

* Use build-args as simple string

* Relaunch CI

* Fix extra arguments

* Launch CI

* Add trivyignore file

* Switch to the main branch of operator-workflows

* Add workload scale integ test

* Kill the leader unit after scaling up

* Address some PR comments

* Remove local_mode and s3_tls option in favor of the extra_env option

* Fix integration tests

* Change ops_test.model into a fixture

* Add a test_user fixture

* Use secrets for localstack access key

* Use requests instead of curl, test content of uploaded file

* Small fixes

* Use tmp_path from pytest instead of a "real" file

* Try to change the model config before launching tests

* Use localstack:1.4

* Don't active pro mode for localstack

* Use IP for s3_config's domain

* Try another fix for s3

* Fixes

* Fix extra-args on CI

* Address PR comments

* Address PR comments

* Fixes

* Fix integ workflow

* Fix the pod matching regex

* Pin k8s dep

* Update remind plugin and add a integ test for it

* Improve code quality

* Reduce scale test to 2 units to avoid potential memory issues

* Try to debug localstack-installation.sh

* Try to mkdir missing dir

* Try fix localstack version to 1.4.0

* Remove debug env

* Try adding a waiting loop

* Address PR comments

* Revert to ops_test.juju("run", ...) instead of app.run()

* Change exception var name

* Address comments

* Trying to debug test

* Fix test

* Try to debug

* Try without regex

* Address comments
  • Loading branch information
nrobinaubertin authored Apr 19, 2023
1 parent 29d0f36 commit c0e5c5d
Show file tree
Hide file tree
Showing 6 changed files with 229 additions and 99 deletions.
24 changes: 12 additions & 12 deletions mattermost.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ RUN mkdir -p /mattermost/data /mattermost/plugins /mattermost/client/plugins &&
;; \
*) \
echo "E: Unknown edition ${edition}! Cannot continue." >&2 ; \
exit 1 ; \
exit 1 ; \
;; \
esac && \
addgroup --gid ${mattermost_gid} mattermost && \
Expand All @@ -106,30 +106,30 @@ RUN if [ "$image_flavour" = canonical ]; then \

# Download and enable third-party plugin
RUN if [ "$image_flavour" = canonical ]; then \
cd /mattermost/plugins && \
set -o pipefail && \
curl -L https://github.com/matterpoll/matterpoll/releases/download/v1.4.0/com.github.matterpoll.matterpoll-1.4.0.tar.gz | tar -xvz ; \
cd /mattermost/plugins && \
set -o pipefail && \
curl -L https://github.com/matterpoll/matterpoll/releases/download/v1.4.0/com.github.matterpoll.matterpoll-1.4.0.tar.gz | tar -xvz ; \
fi

# Download and enable third-party plugin
RUN if [ "$image_flavour" = canonical ]; then \
cd /mattermost/plugins && \
set -o pipefail && \
curl -L https://github.com/moussetc/mattermost-plugin-giphy/releases/download/v2.1.1/com.github.moussetc.mattermost.plugin.giphy-2.1.1.tar.gz | tar -xvz ; \
cd /mattermost/plugins && \
set -o pipefail && \
curl -L https://github.com/moussetc/mattermost-plugin-giphy/releases/download/v2.1.1/com.github.moussetc.mattermost.plugin.giphy-2.1.1.tar.gz | tar -xvz ; \
fi

# Download and enable third-party plugin
RUN if [ "$image_flavour" = canonical ]; then \
cd /mattermost/plugins && \
set -o pipefail && \
curl -L https://github.com/scottleedavis/mattermost-plugin-remind/releases/download/v0.4.5/com.github.scottleedavis.mattermost-plugin-remind-0.4.5.tar.gz | tar -xvz ; \
cd /mattermost/plugins && \
set -o pipefail && \
curl -L https://github.com/scottleedavis/mattermost-plugin-remind/releases/download/v1.0.0/com.github.scottleedavis.mattermost-plugin-remind-1.0.0.tar.gz | tar -xvz ; \
fi

# Canonical's custom webapp
COPY --from=canonical_flavour_builder /mattermost-webapp/dist/. /canonical_flavour_tmp/
RUN if [ "$image_flavour" = canonical ]; then \
rm -rf /mattermost/client && \
cp -r /canonical_flavour_tmp/. /mattermost/client ; \
rm -rf /mattermost/client && \
cp -r /canonical_flavour_tmp/. /mattermost/client ; \
fi

RUN rm -rf /canonical_flavour_tmp
Expand Down
5 changes: 3 additions & 2 deletions src/environment.py
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,7 @@ def generate(config: dict, app_name: str, site_url: str, db_uri: str) -> dict:
}
)

# replace some configurations with canonical defaults if chosen so
if config["use_canonical_defaults"]:
env.update(CANONICAL_DEFAULTS)

Expand All @@ -274,8 +275,8 @@ def generate(config: dict, app_name: str, site_url: str, db_uri: str) -> dict:
# Update env with provided extra_env
try:
env.update(json.loads(config["extra_env"]))
except json.JSONDecodeError:
raise json.jSONDecodeError("extra_env is not valid JSON")
except json.JSONDecodeError as exc:
raise json.JSONDecodeError("extra_env is not valid JSON", exc.doc, exc.pos)

# make sure to convert all values to str
env = {env_name: str(env_value) for env_name, env_value in env.items()}
Expand Down
92 changes: 74 additions & 18 deletions tests/integration/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,23 @@
import json
import logging
import secrets
import time
from pathlib import Path
from urllib.parse import urlparse

import kubernetes
import ops
import pytest_asyncio
import requests
import yaml
from boto3 import client
from botocore.config import Config
from ops.model import ActiveStatus, Application
from pytest import FixtureRequest, fixture
from pytest_operator.plugin import OpsTest

import utils

logger = logging.getLogger(__name__)


Expand All @@ -44,15 +48,6 @@ def mattermost_image(request):
return request.config.getoption("--mattermost-image")


@fixture(scope="module")
def test_user():
"""Create login informations for a test user.
Return a dict with the users informations
"""
return {"login_id": "[email protected]", "password": secrets.token_hex()}


@pytest_asyncio.fixture(scope="module", name="model")
async def model_fixture(ops_test: OpsTest) -> ops.model.Model:
"""Provide current test model."""
Expand All @@ -79,10 +74,18 @@ async def app(
application = await model.deploy(charm, application_name=app_name, series="focal")
await model.wait_for_idle()

# change the image that will be used for the mattermost container
# Change the image that will be used for the mattermost container
# and use some common test configuration extra env variables
await application.set_config(
{
"mattermost_image_path": mattermost_image,
"extra_env": json.dumps(
{
"MM_FILESETTINGS_AMAZONS3SSL": "false",
"MM_SERVICESETTINGS_ENABLELOCALMODE": "true",
"MM_SERVICESETTINGS_LOCALMODESOCKETLOCATION": "/tmp/mattermost.socket",
}
),
}
)
await model.wait_for_idle()
Expand All @@ -93,22 +96,75 @@ async def app(
# mypy doesn't see that ActiveStatus has a name
await model.wait_for_idle(status=ActiveStatus.name) # type: ignore

# test that the application is online
for _ in range(10):
if await utils.is_mattermost_reachable(
await utils.get_mattermost_ip(ops_test, application)
):
break
time.sleep(10)

assert await utils.is_mattermost_reachable(
await utils.get_mattermost_ip(ops_test, application)
)
yield application


@pytest_asyncio.fixture(scope="module")
async def mattermost_ip(
async def test_entities(
ops_test: OpsTest,
app: Application,
):
"""Get the IP address of the first unit of mattermost.
"""Create some usual test entities to be used in integration tests."""
mattermost_ip = await utils.get_mattermost_ip(ops_test, app)

Return the IP address of a mattermost unit.
"""
unit_informations = json.loads(
(await ops_test.juju("show-unit", app.units[0].name, "--format", "json"))[1]
test_user = {"login_id": "[email protected]", "password": secrets.token_hex()}

# create a user
cmd = (
f"MMCTL_LOCAL_SOCKET_PATH=/tmp/mattermost.socket /mattermost/bin/mmctl"
f" --local user create --email {test_user['login_id']} --username test"
f" --password {test_user['password']}"
)
await ops_test.juju("run", "--application", app.name, cmd)

# login to the API
response = requests.post(
f"http://{mattermost_ip}:8065/api/v4/users/login", data=json.dumps(test_user), timeout=10
)
return unit_informations[app.units[0].name]["address"]
token = response.headers["Token"]
headers = {"authorization": f"Bearer {response.headers['Token']}"}

# create a team
data = {"name": "test", "display_name": "test", "type": "O"}
response = requests.post(
f"http://{mattermost_ip}:8065/api/v4/teams",
data=json.dumps(data),
headers=headers,
timeout=10,
)
team = response.json()

# create a channel
data = {"team_id": team["id"], "name": "test", "display_name": "test", "type": "O"}
response = requests.post(
f"http://{mattermost_ip}:8065/api/v4/channels",
data=json.dumps(data),
headers=headers,
timeout=10,
)
channel = response.json()

yield {
"token": token,
"headers": headers,
"user": {
"email": test_user["login_id"],
"password": test_user["password"],
},
"team": team,
"channel": channel,
}


@fixture(scope="module")
Expand Down Expand Up @@ -180,7 +236,7 @@ def localstack_s3_client(localstack_s3_config: dict) -> client:

@fixture(scope="module", name="kube_config")
def kube_config_fixture(request: FixtureRequest):
"""The Kubernetes cluster configuration file."""
"""Return the Kubernetes cluster configuration file."""
kube_config = request.config.getoption("--kube-config")
assert kube_config, (
"The Kubernetes config file path should not be empty, "
Expand Down
10 changes: 7 additions & 3 deletions tests/integration/localstack-installation.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,13 @@

pip install pip --upgrade
pip install pyopenssl --upgrade
pip install localstack # install LocalStack cli
docker pull localstack/localstack # Make sure to pull the latest version of the image
ACTIVATE_PRO=0 EDGE_BIND_HOST=0.0.0.0 localstack start -d # Start LocalStack in the background (binding to all host ip)
pip install localstack==1.4.0
docker pull localstack/localstack:1.4.0
ACTIVATE_PRO=0 \
EDGE_BIND_HOST=0.0.0.0 \
EDGE_PORT=4566 \
IMAGE_NAME=localstack/localstack:1.4.0 \
localstack start --docker -d # Start LocalStack in the background (binding to all host ip)
echo "Waiting for LocalStack startup..." # Wait 30 seconds for the LocalStack container
localstack wait -t 30 # to become ready before timing out
echo "Startup complete"
Loading

0 comments on commit c0e5c5d

Please sign in to comment.