Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download button - in preferences for downloading policies #2546

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions docs/resource-validation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,43 @@ Policies can also reference rules in different rule sets as long as they are loa

To see the full specifications, check the [example rule-set](../../examples/resource-validation/rule-set.yaml).

### Alternative format: Custom Rules

Rules and policies can also be given using [datree's Custom Rules](https://hub.datree.io/custom-rules/custom-rules-overview) format as part of a rule set.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link doesn't work (returns 404).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@IwonaLanger, thanks for checking this. I also did a check last week and can confirm that the page was indeed a 404 page temporarily. Now it is up again...

As it is an external link, this is not under our control. Should we keep it like it is? Alternatively, we could remove the link completely or only link to the entry page hub.datree.io. But I think a deep link would be more helpful.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Akatuoro Let's keep the link. If it was just a temporary failure, there's no need to remove it.


```yaml
apiVersion: v1
policies:
- name: TestPolicy
isDefault: true
rules:
- identifier: TEST
messageOnFailure: This is a test rule
customRules:
- identifier: TEST
name: This is a test rule
defaultMessageOnFailure: This is a test rule
schema:
required:
- test
properties:
test:
type: object
required:
- hello
properties:
hello:
type: string
enum:
- kyma
```

## Download policies

You can download the currently enabled policies.

In the user preferences, go to **Clusters > Resource Validation**. Click on **Download** in the **Enabled Policies** section.

## Scan the cluster

With the **CLUSTER_VALIDATION** [feature flag](../features.md) enabled, you can use these rules to scan existing resources in your cluster.
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
28 changes: 28 additions & 0 deletions examples/resource-validation/custom-policies.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# datree 'policy as code' format
# Can be validated using datree command:
# datree test --ignore-missing-schemas --no-record --policy-config examples/resource-validation/custom-policies.yaml examples/resource-validation/pod.yaml

apiVersion: v1
policies:
- name: TestPolicy
isDefault: true
rules:
- identifier: TEST
messageOnFailure: This is a test rule
customRules:
- identifier: TEST
name: This is a test rule
defaultMessageOnFailure: This is a test rule
schema:
required:
- test
properties:
test:
type: object
required:
- hello
properties:
hello:
type: string
enum:
- kyma
3 changes: 3 additions & 0 deletions public/i18n/en.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,7 @@ common:
copy: Copy
create: Create
delete: Delete
download: Download
edit: Edit
edit-yaml: Edit YAML
generate-name: Generate
Expand Down Expand Up @@ -1334,6 +1335,8 @@ settings:
customize: Customize
validation-disabled: Resource validation is disabled
no-policies-found: No policies found
download-error: Error while downloading rule set
rule-set-missing: No rule set found
interaction:
title: Cluster Interaction
interface:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,6 @@ rules:
properties:
runAsNonRoot:
enum:
- null
- null
- true
validSpec:
Expand Down Expand Up @@ -472,7 +471,6 @@ rules:
properties:
type:
enum:
- null
- null
- RuntimeDefault
- Localhost
Expand Down
Original file line number Diff line number Diff line change
@@ -1,17 +1,24 @@
import { useTranslation } from 'react-i18next';
import jsyaml from 'js-yaml';
import { saveAs } from 'file-saver';
import { useRecoilState, useRecoilValue } from 'recoil';
import { Button, LayoutPanel, Switch } from 'fundamental-react';
import {
getExtendedValidateResourceState,
validateResourcesState,
} from 'state/preferences/validateResourcesAtom';
import { validationSchemasState } from 'state/validationSchemasAtom';
import { useMemo } from 'react';
import { useCallback, useMemo } from 'react';
import { GenericList } from 'shared/components/GenericList/GenericList';

import './ResourceValidationSettings.scss';
import { useFeature } from 'hooks/useFeature';
import { ValidationFeatureConfig } from 'state/validationEnabledSchemasAtom';
import {
convertPoliciesForDatree,
usePolicySet,
ValidationFeatureConfig,
} from 'state/validationEnabledSchemasAtom';
import { useNotification } from 'shared/contexts/NotificationContext';

export default function ResourceValidationSettings() {
const { t } = useTranslation();
Expand Down Expand Up @@ -39,6 +46,34 @@ export default function ResourceValidationSettings() {
[validationSchemas],
);

const policySet = usePolicySet();
const notification = useNotification();

const download = useCallback(() => {
if (validationSchemas) {
try {
const customPolicyDefinition = convertPoliciesForDatree(
validationSchemas,
policySet,
);
const kubeconfigYaml = jsyaml.dump(customPolicyDefinition);
const blob = new Blob([kubeconfigYaml], {
type: 'application/yaml;charset=utf-8',
});
saveAs(blob, `customPolicies.yaml`);
} catch (e) {
console.error(e);
notification.notifyError({
content: t('settings.clusters.resourcesValidation.download-error'),
});
}
} else {
notification.notifyError({
content: t('settings.clusters.resourcesValidation.rule-set-missing'),
});
}
}, [validationSchemas, policySet, notification, t]);

const policyList = useMemo(() => {
const selectedPolicySet = selectedPolicies.reduce(
(agg, name) => agg.add(name),
Expand Down Expand Up @@ -182,6 +217,14 @@ export default function ResourceValidationSettings() {
{t('settings.clusters.resourcesValidation.reset')}
</Button>
)}
<Button
option="transparent"
glyph="download"
className="fd-margin-begin--sm"
onClick={download}
>
{t('common.buttons.download')}
</Button>
</>
}
searchSettings={{
Expand Down
81 changes: 81 additions & 0 deletions src/state/validationEnabledSchemasAtom.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import {
import {
emptyValidationSchema,
getEnabledRules,
getResolvedPolicy,
DatreeCustomRule,
ValidationSchema,
validationSchemasState,
} from './validationSchemasAtom';
Expand All @@ -22,6 +24,85 @@ export type ValidationFeatureConfig = {
};
};

type DatreeCustomPolicies = {
apiVersion: string;
policies: {
name: string;
isDefault: boolean;
rules: {
identifier: string;
messageOnFailure: string;
}[];
}[];
customRules: DatreeCustomRule[];
};

/**
* The datree format differs between its own rules and the custom rules via 'policy as code'.
* This function converts the rules of a given rule set and selected policies into a format,
* which can be used in datree via the --policy-config option.
*/
export const convertPoliciesForDatree = (
validationSchemas: ValidationSchema,
policySet: Set<string>,
): DatreeCustomPolicies => {
const enabledSchemas = getValidationEnabledSchemas(
validationSchemas,
policySet,
);

const customRules = enabledSchemas.rules.map(
({ uniqueName, name, schema, messageOnFailure }) => {
return {
identifier: uniqueName,
name: name ?? uniqueName,
defaultMessageOnFailure:
messageOnFailure ?? `Rule ${uniqueName} failed`,
schema,
};
},
);

const policies = enabledSchemas.policies.map(policy => {
const resolvedPolicy = getResolvedPolicy(
policy,
validationSchemas.policies,
);
const rules = resolvedPolicy.rules.map(rule => {
if (typeof rule === 'string') {
const messageOnFailure =
customRules.find(({ identifier }) => identifier)
?.defaultMessageOnFailure || `Rule ${rule} failed`;
return {
identifier: rule,
messageOnFailure,
};
} else {
return {
identifier: rule.identifier,
messageOnFailure: rule.messageOnFailure ?? `Rule ${rule} failed`,
};
}
});

return {
name: resolvedPolicy.name,
isDefault: false,
rules,
};
});
if (policies.length > 0) {
policies[0].isDefault = true;
}

const customPolicyDefinition = {
apiVersion: 'v1',
policies,
customRules,
};
return customPolicyDefinition;
};

const getEnabledPolicyNames = (
validationFeature: ValidationFeatureConfig,
validationPreferences: ExtendedValidateResources,
Expand Down
58 changes: 51 additions & 7 deletions src/state/validationSchemasAtom.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,25 @@ import { FetchFn } from 'shared/hooks/BackendAPI/useFetch';

type Rule = {
uniqueName: string;
name?: string;
messageOnFailure?: string;
documentationUrl?: string;
category?: string;
policies?: ValidationPolicy[];
schema: JSONSchema4;
};

export type DatreeCustomRule = {
identifier: string;
name: string;
defaultMessageOnFailure: string;
schema: JSONSchema4;
};

type ValidationConfig = {
rules?: Array<Rule>;
policies?: Array<ValidationPolicy>;
customRules?: Array<DatreeCustomRule>;
};

export type ValidationSchema = {
Expand All @@ -41,6 +50,7 @@ type RuleReference =
| string
| {
identifier: string;
messageOnFailure?: string;
};

type ValidationPolicy = {
Expand All @@ -50,6 +60,20 @@ type ValidationPolicy = {
rules: Array<RuleReference>;
};

const convertCustomRules = (customRules?: DatreeCustomRule[]): Rule[] => {
if (!customRules) return [];
return customRules?.map(
({ identifier, name, defaultMessageOnFailure, schema }) => {
return {
uniqueName: identifier,
name,
messageOnFailure: defaultMessageOnFailure,
schema,
};
},
);
};

const fetchBaseValidationConfig = async (): Promise<ValidationConfig[]> => {
try {
const response = await fetch(`/resource-validation/rule-set.yaml`);
Expand Down Expand Up @@ -104,9 +128,13 @@ const fetchValidationConfig = async (
permissionSet,
);

const rules = [...validationConfig, ...configFromConfigMap].flatMap(
schema => schema.rules ?? [],
);
const rules = [
...validationConfig,
...configFromConfigMap,
].flatMap(schema => [
...(schema.rules ?? []),
...convertCustomRules(schema.customRules),
]);

const policies = [...validationConfig, ...configFromConfigMap].flatMap(
schema => schema.policies ?? [],
Expand All @@ -115,6 +143,14 @@ const fetchValidationConfig = async (
return { rules, policies };
};

const getPolicyMap = (policies?: ValidationPolicy[]) => {
return (
policies?.reduce((agg, policy) => {
return agg.set(policy.name, policy);
}, new Map()) ?? (new Map() as Map<string, ValidationPolicy>)
);
};

const getRuleKey = (rule: RuleReference) => {
return typeof rule === 'string' ? rule : rule.identifier;
};
Expand Down Expand Up @@ -152,6 +188,17 @@ const resolveRulesForPolicy = (
return [...rules, ...additionalRules];
};

export const getResolvedPolicy = (
policy: ValidationPolicy,
allPolicies: ValidationPolicy[],
) => {
const policyMap = getPolicyMap(allPolicies);
return {
...policy,
rules: resolveRulesForPolicy(policy, policyMap),
};
};

export const getEnabledRules = (
rules: Rule[],
selectedPolicies: ValidationPolicy[],
Expand All @@ -162,10 +209,7 @@ export const getEnabledRules = (
{},
) as { [key: string]: Rule };

const policyMap =
allPolicies?.reduce((agg, policy) => {
return agg.set(policy.name, policy);
}, new Map()) ?? (new Map() as Map<string, ValidationPolicy>);
const policyMap = getPolicyMap(allPolicies);

const enabledRulesByName = selectedPolicies.reduce((agg, policy) => {
const policyRules = resolveRulesForPolicy(policy, policyMap);
Expand Down
Loading
Loading