Skip to content

Commit

Permalink
Merge pull request #3076 from grafana/dev
Browse files Browse the repository at this point in the history
Merge dev to main
  • Loading branch information
mderynck authored Sep 27, 2023
2 parents 389a887 + 9126f21 commit 65bca4a
Show file tree
Hide file tree
Showing 67 changed files with 1,887 additions and 458 deletions.
16 changes: 15 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,20 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased
## v1.3.39 (2023-09-27)

### Added

- Presets for webhooks @mderynck ([#2996](https://github.com/grafana/oncall/pull/2996))
- Add `enable_web_overrides` option to schedules public API ([#3062](https://github.com/grafana/oncall/pull/3062))

### Fixed

- Fix regression in public actions endpoint handling user field by @mderynck ([#3053](https://github.com/grafana/oncall/pull/3053))

### Changed

- Rework how users are fetched from DB when getting users from schedules ical representation ([#3067](https://github.com/grafana/oncall/pull/3067))

## v1.3.38 (2023-09-19)

Expand All @@ -20,6 +33,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added

- Notify user via Slack/mobile push-notification when their shift swap request is taken by @joeyorlando ([#2992](https://github.com/grafana/oncall/pull/2992))
- Unify breadcrumbs behaviour with other Grafana Apps and main core# ([1906](https://github.com/grafana/oncall/issues/1906))

### Changed

Expand Down
12 changes: 6 additions & 6 deletions docs/docs.mk
Original file line number Diff line number Diff line change
Expand Up @@ -80,35 +80,35 @@ docs-pull: ## Pull documentation base image.

make-docs: ## Fetch the latest make-docs script.
make-docs:
if [[ ! -f "$(PWD)/make-docs" ]]; then
if [[ ! -f "$(CURDIR)/make-docs" ]]; then
echo 'WARN: No make-docs script found in the working directory. Run `make update` to download it.' >&2
exit 1
fi

.PHONY: docs
docs: ## Serve documentation locally, which includes pulling the latest `DOCS_IMAGE` (default: `grafana/docs-base:latest`) container image. See also `docs-no-pull`.
docs: docs-pull make-docs
$(PWD)/make-docs $(PROJECTS)
$(CURDIR)/make-docs $(PROJECTS)

.PHONY: docs-no-pull
docs-no-pull: ## Serve documentation locally without pulling the `DOCS_IMAGE` (default: `grafana/docs-base:latest`) container image.
docs-no-pull: make-docs
$(PWD)/make-docs $(PROJECTS)
$(CURDIR)/make-docs $(PROJECTS)

.PHONY: docs-debug
docs-debug: ## Run Hugo web server with debugging enabled. TODO: support all SERVER_FLAGS defined in website Makefile.
docs-debug: make-docs
WEBSITE_EXEC='hugo server --bind 0.0.0.0 --port 3002 --debug' $(PWD)/make-docs $(PROJECTS)
WEBSITE_EXEC='hugo server --bind 0.0.0.0 --port 3002 --debug' $(CURDIR)/make-docs $(PROJECTS)

.PHONY: doc-validator
doc-validator: ## Run doc-validator on the entire docs folder.
doc-validator: make-docs
DOCS_IMAGE=$(DOC_VALIDATOR_IMAGE) $(PWD)/make-docs $(PROJECTS)
DOCS_IMAGE=$(DOC_VALIDATOR_IMAGE) $(CURDIR)/make-docs $(PROJECTS)

.PHONY: vale
vale: ## Run vale on the entire docs folder.
vale: make-docs
DOCS_IMAGE=$(VALE_IMAGE) $(PWD)/make-docs $(PROJECTS)
DOCS_IMAGE=$(VALE_IMAGE) $(CURDIR)/make-docs $(PROJECTS)

.PHONY: update
update: ## Fetch the latest version of this Makefile and the `make-docs` script from Writers' Toolkit.
Expand Down
1 change: 1 addition & 0 deletions docs/sources/oncall-api-reference/schedules.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ The above command returns JSON structured in the following way:
| `time_zone` | No | Optional | Schedule time zone. Is used for manually added on-call shifts in Schedules with type `calendar`. Default time zone is `UTC`. For more information about time zones, see [time zones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). |
| `ical_url_primary` | No | If type = `ical` | URL of external iCal calendar for schedule with type `ical`. |
| `ical_url_overrides` | No | Optional | URL of external iCal calendar for schedule with any type. Events from this calendar override events from primary calendar or from on-call shifts. |
| `enable_web_overrides` | No | Optional | Whether to enable web overrides or not. Setting specific for API/Terraform based schedules (`calendar` type). |
| `slack` | No | Optional | Dictionary with Slack-specific settings for a schedule. Includes `channel_id` and `user_group_id` fields, that take a channel ID and a user group ID from Slack. |
| `shifts` | No | Optional | List of shifts. Used for manually added on-call shifts in Schedules with type `calendar`. |

Expand Down
6 changes: 4 additions & 2 deletions docs/sources/outgoing-webhooks/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,10 @@ Jinja2 templates to customize the request being sent.
## Creating an outgoing webhook

To create an outgoing webhook navigate to **Outgoing Webhooks** and click **+ Create**. On this screen outgoing
webhooks can be viewed, edited and deleted. To create the outgoing webhook populate the required fields and
click **Create Webhook**
webhooks can be viewed, edited and deleted. To create the outgoing webhook click **New Outgoing Webhook** and then
select a preset based on what you want to do. A simple webhook will POST alert group data as a selectable escalation
step to the specified url. If you require more customization use the advanced webhook which provides all of the
fields described below.

### Outgoing webhook fields

Expand Down
81 changes: 74 additions & 7 deletions engine/apps/api/serializers/webhook.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
from rest_framework.validators import UniqueTogetherValidator

from apps.webhooks.models import Webhook, WebhookResponse
from apps.webhooks.models.webhook import WEBHOOK_FIELD_PLACEHOLDER
from apps.webhooks.models.webhook import PUBLIC_WEBHOOK_HTTP_METHODS, WEBHOOK_FIELD_PLACEHOLDER
from apps.webhooks.presets.preset_options import WebhookPresetOptions
from common.api_helpers.custom_fields import TeamPrimaryKeyRelatedField
from common.api_helpers.utils import CurrentOrganizationDefault, CurrentTeamDefault, CurrentUserDefault
from common.jinja_templater import apply_jinja_template
Expand All @@ -31,9 +32,9 @@ class WebhookSerializer(serializers.ModelSerializer):
organization = serializers.HiddenField(default=CurrentOrganizationDefault())
team = TeamPrimaryKeyRelatedField(allow_null=True, default=CurrentTeamDefault())
user = serializers.HiddenField(default=CurrentUserDefault())
trigger_type = serializers.CharField(required=True)
forward_all = serializers.BooleanField(allow_null=True, required=False)
last_response_log = serializers.SerializerMethodField()
trigger_type = serializers.CharField(allow_null=True)
trigger_type_name = serializers.SerializerMethodField()

class Meta:
Expand All @@ -59,11 +60,8 @@ class Meta:
"trigger_type_name",
"last_response_log",
"integration_filter",
"preset",
]
extra_kwargs = {
"name": {"required": True, "allow_null": False, "allow_blank": False},
"url": {"required": True, "allow_null": False, "allow_blank": False},
}

validators = [UniqueTogetherValidator(queryset=Webhook.objects.all(), fields=["name", "organization"])]

Expand All @@ -78,6 +76,16 @@ def to_representation(self, instance):
def to_internal_value(self, data):
webhook = self.instance

# Some fields are conditionally required, add none values for missing required fields
if webhook and webhook.preset and "preset" not in data:
data["preset"] = webhook.preset
for key in ["url", "http_method", "trigger_type"]:
if key not in data:
if self.instance:
data[key] = getattr(self.instance, key)
else:
data[key] = None

# If webhook is being copied instance won't exist to copy values from
if not webhook and "id" in data:
webhook = Webhook.objects.get(
Expand Down Expand Up @@ -111,10 +119,29 @@ def validate_headers(self, headers):
return self._validate_template_field(headers)

def validate_url(self, url):
if self.is_field_controlled("url"):
return url

if not url:
return None
raise serializers.ValidationError(detail="This field is required.")
return self._validate_template_field(url)

def validate_http_method(self, http_method):
if self.is_field_controlled("http_method"):
return http_method

if http_method not in PUBLIC_WEBHOOK_HTTP_METHODS:
raise serializers.ValidationError(detail=f"This field must be one of {PUBLIC_WEBHOOK_HTTP_METHODS}.")
return http_method

def validate_trigger_type(self, trigger_type):
if self.is_field_controlled("trigger_type"):
return trigger_type

if not trigger_type or int(trigger_type) not in Webhook.ALL_TRIGGER_TYPES:
raise serializers.ValidationError(detail="This field is required.")
return trigger_type

def validate_data(self, data):
if not data:
return None
Expand All @@ -125,6 +152,29 @@ def validate_forward_all(self, data):
return False
return data

def validate_preset(self, preset):
if self.instance and self.instance.preset != preset:
raise serializers.ValidationError(detail="This field once set cannot be modified.")

if preset:
if preset not in WebhookPresetOptions.WEBHOOK_PRESETS:
raise serializers.ValidationError(detail=f"{preset} is not a valid preset id.")

preset_metadata = WebhookPresetOptions.WEBHOOK_PRESETS[preset].metadata
for controlled_field in preset_metadata.controlled_fields:
if controlled_field in self.initial_data:
if self.instance:
if self.initial_data[controlled_field] != getattr(self.instance, controlled_field):
raise serializers.ValidationError(
detail=f"{controlled_field} is controlled by preset, cannot update"
)
elif self.initial_data[controlled_field] is not None:
raise serializers.ValidationError(
detail=f"{controlled_field} is controlled by preset, cannot create"
)

return preset

def get_last_response_log(self, obj):
return WebhookResponseSerializer(obj.responses.all().last()).data

Expand All @@ -133,3 +183,20 @@ def get_trigger_type_name(self, obj):
if obj.trigger_type is not None:
trigger_type_name = Webhook.TRIGGER_TYPES[int(obj.trigger_type)][1]
return trigger_type_name

def is_field_controlled(self, field_name):
if self.instance:
if not self.instance.preset:
return False
elif "preset" not in self.initial_data:
return False

preset_id = self.instance.preset if self.instance else self.initial_data["preset"]
if preset_id:
if preset_id not in WebhookPresetOptions.WEBHOOK_PRESETS:
raise serializers.ValidationError(detail=f"unknown preset {preset_id} referenced")

preset = WebhookPresetOptions.WEBHOOK_PRESETS[preset_id]
if field_name not in preset.metadata.controlled_fields:
return False
return True
2 changes: 1 addition & 1 deletion engine/apps/api/tests/test_schedules.py
Original file line number Diff line number Diff line change
Expand Up @@ -2081,7 +2081,7 @@ def test_get_schedule_on_call_now(
client = APIClient()
url = reverse("api-internal:schedule-list")
with patch(
"apps.schedules.models.on_call_schedule.OnCallScheduleQuerySet.get_oncall_users",
"apps.api.views.schedule.get_oncall_users_for_multiple_schedules",
return_value={schedule.pk: [user]},
):
response = client.get(url, format="json", **make_user_auth_headers(user, token))
Expand Down
161 changes: 161 additions & 0 deletions engine/apps/api/tests/test_webhook_presets.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
import pytest
from django.urls import reverse
from rest_framework import status
from rest_framework.test import APIClient

from apps.webhooks.models import Webhook
from apps.webhooks.models.webhook import WEBHOOK_FIELD_PLACEHOLDER
from apps.webhooks.tests.test_webhook_presets import (
TEST_WEBHOOK_LOGO,
TEST_WEBHOOK_PRESET_CONTROLLED_FIELDS,
TEST_WEBHOOK_PRESET_DESCRIPTION,
TEST_WEBHOOK_PRESET_ID,
TEST_WEBHOOK_PRESET_NAME,
TEST_WEBHOOK_PRESET_URL,
)


@pytest.mark.django_db
def test_get_webhook_preset_options(
make_organization_and_user_with_plugin_token, webhook_preset_api_setup, make_user_auth_headers
):
organization, user, token = make_organization_and_user_with_plugin_token()
client = APIClient()
url = reverse("api-internal:webhooks-preset-options")

response = client.get(url, format="json", **make_user_auth_headers(user, token))

assert response.status_code == status.HTTP_200_OK
assert response.data[0]["id"] == TEST_WEBHOOK_PRESET_ID
assert response.data[0]["name"] == TEST_WEBHOOK_PRESET_NAME
assert response.data[0]["logo"] == TEST_WEBHOOK_LOGO
assert response.data[0]["description"] == TEST_WEBHOOK_PRESET_DESCRIPTION
assert response.data[0]["controlled_fields"] == TEST_WEBHOOK_PRESET_CONTROLLED_FIELDS


@pytest.mark.django_db
def test_create_webhook_from_preset(
make_organization_and_user_with_plugin_token, webhook_preset_api_setup, make_user_auth_headers
):
organization, user, token = make_organization_and_user_with_plugin_token()
client = APIClient()
url = reverse("api-internal:webhooks-list")

data = {
"name": "the_webhook",
"trigger_type": Webhook.TRIGGER_ALERT_GROUP_CREATED,
"team": None,
"password": "secret_password",
"preset": TEST_WEBHOOK_PRESET_ID,
}
response = client.post(url, data, format="json", **make_user_auth_headers(user, token))
webhook = Webhook.objects.get(public_primary_key=response.data["id"])

expected_response = data | {
"id": webhook.public_primary_key,
"url": TEST_WEBHOOK_PRESET_URL,
"data": organization.org_title,
"username": None,
"password": WEBHOOK_FIELD_PLACEHOLDER,
"authorization_header": None,
"forward_all": True,
"headers": None,
"http_method": "GET",
"integration_filter": None,
"is_webhook_enabled": True,
"is_legacy": False,
"last_response_log": {
"request_data": "",
"request_headers": "",
"timestamp": None,
"content": "",
"status_code": None,
"request_trigger": "",
"url": "",
"event_data": "",
},
"trigger_template": None,
"trigger_type": str(data["trigger_type"]),
"trigger_type_name": "Alert Group Created",
}

assert response.status_code == status.HTTP_201_CREATED
assert response.json() == expected_response
assert webhook.password == data["password"]


@pytest.mark.django_db
def test_invalid_create_webhook_with_preset(
make_organization_and_user_with_plugin_token, webhook_preset_api_setup, make_user_auth_headers
):
organization, user, token = make_organization_and_user_with_plugin_token()
client = APIClient()
url = reverse("api-internal:webhooks-list")

data = {
"name": "the_webhook",
"trigger_type": Webhook.TRIGGER_ALERT_GROUP_CREATED,
"url": "https://test12345.com",
"preset": TEST_WEBHOOK_PRESET_ID,
}
response = client.post(url, data, format="json", **make_user_auth_headers(user, token))
assert response.status_code == status.HTTP_400_BAD_REQUEST
assert response.json()["preset"][0] == "url is controlled by preset, cannot create"


@pytest.mark.django_db
def test_update_webhook_from_preset(
make_organization_and_user_with_plugin_token, webhook_preset_api_setup, make_user_auth_headers, make_custom_webhook
):
organization, user, token = make_organization_and_user_with_plugin_token()
webhook = make_custom_webhook(
name="the_webhook",
organization=organization,
trigger_type=Webhook.TRIGGER_ALERT_GROUP_CREATED,
preset=TEST_WEBHOOK_PRESET_ID,
)

client = APIClient()
url = reverse("api-internal:webhooks-detail", kwargs={"pk": webhook.public_primary_key})

data = {
"name": "the_webhook 2",
}
response = client.put(url, data, format="json", **make_user_auth_headers(user, token))
assert response.status_code == status.HTTP_200_OK
assert response.json()["name"] == data["name"]

webhook.refresh_from_db()
assert webhook.name == data["name"]
assert webhook.url == TEST_WEBHOOK_PRESET_URL
assert webhook.http_method == "GET"
assert webhook.data == organization.org_title


@pytest.mark.django_db
def test_invalid_update_webhook_from_preset(
make_organization_and_user_with_plugin_token, webhook_preset_api_setup, make_user_auth_headers, make_custom_webhook
):
organization, user, token = make_organization_and_user_with_plugin_token()
webhook = make_custom_webhook(
name="the_webhook",
organization=organization,
trigger_type=Webhook.TRIGGER_ALERT_GROUP_CREATED,
preset=TEST_WEBHOOK_PRESET_ID,
)

client = APIClient()
url = reverse("api-internal:webhooks-detail", kwargs={"pk": webhook.public_primary_key})

data = {
"preset": "some_other_preset",
}
response = client.put(url, data, format="json", **make_user_auth_headers(user, token))
assert response.status_code == status.HTTP_400_BAD_REQUEST
assert response.json()["preset"][0] == "This field once set cannot be modified."

data = {
"data": "some_other_data",
}
response = client.put(url, data, format="json", **make_user_auth_headers(user, token))
assert response.status_code == status.HTTP_400_BAD_REQUEST
Loading

0 comments on commit 65bca4a

Please sign in to comment.