Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Add databricks_function resource #4189

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

[Feature] Add databricks_function resource #4189

wants to merge 14 commits into from

Conversation

dgomez04
Copy link
Contributor

@dgomez04 dgomez04 commented Nov 2, 2024 •
edited by alexott
Loading

Changes

Introduces databricks_function resource—allowing users to create and manage UDFs within Terraform. work in progress...

Closes #4074

Tests

  • make test run locally
  • relevant change in docs/ folder
  • covered with integration tests in internal/acceptance
  • relevant acceptance tests are passing
  • using Go SDK

@dgomez04 dgomez04 requested review from a team as code owners November 2, 2024 02:35
@dgomez04 dgomez04 requested review from hectorcast-db and removed request for a team November 2, 2024 02:35
@dgomez04 dgomez04 changed the title [FEATURE] Add databricks_function resource [FEATURE] WIP Add databricks_function resource Nov 2, 2024
@dgomez04 dgomez04 changed the title [FEATURE] WIP Add databricks_function resource [Feature] [WIP] Add databricks_function resource Nov 2, 2024
@ian-norris-ncino
Copy link

Hello, is there any timeline on when this could make it into the provider? Would love to have this soon!

@dgomez04
Copy link
Contributor Author

dgomez04 commented Nov 4, 2024

Hi @ian-norris-ncino. I'm aiming to have this integrated into the provider by the end of the month, depending on my availability. I'll keep you posted.

@dgomez04 dgomez04 changed the title [Feature] [WIP] Add databricks_function resource [Feature] Add databricks_function resource Nov 7, 2024
@alexott alexott requested review from rauchy and tanmay-db November 12, 2024 14:06
alexott
alexott requested changes Nov 16, 2024
View reviewed changes
docs/resources/function.md Outdated
name = "calculate_bmi"
catalog_name = databricks_catalog.sandbox.name
schema_name = databricks_schema.functions.name
input_params = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be easier for people to specify input_param as separate blocks, like we do in other resources

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although doc says that input_params is a block, and then there are parameters inside: https://docs.databricks.com/api/workspace/functions/create#input_params

docs/resources/function.md Outdated Show resolved Hide resolved
@eng-dev-ecosystem-bot
Copy link
Collaborator

Test Details: go/deco-tests/11869831592

tanmay-db
tanmay-db reviewed Nov 27, 2024
View reviewed changes
Copy link
Contributor

@tanmay-db tanmay-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @dgomez04, thanks for the PR, left some comments.

internal/providers/pluginfw/resources/catalog/resource_function.go Outdated Show resolved Hide resolved
internal/providers/pluginfw/resources/catalog/resource_function.go Outdated Show resolved Hide resolved
internal/providers/pluginfw/resources/catalog/resource_function.go Outdated Show resolved Hide resolved
internal/providers/pluginfw/resources/catalog/resource_function.go Outdated Show resolved Hide resolved
internal/providers/pluginfw/resources/catalog/resource_function_acc_test.go Outdated Show resolved Hide resolved
mgyucht
mgyucht requested changes Dec 20, 2024
View reviewed changes
Copy link
Contributor

@mgyucht mgyucht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking nice! Some changes needed for the next iteration, but I think we're not far from the final state.

docs/resources/function.md Outdated Show resolved Hide resolved
docs/resources/function.md Outdated Show resolved Hide resolved
internal/providers/pluginfw/pluginfw_rollout_utils.go Outdated Show resolved Hide resolved
internal/providers/pluginfw/pluginfw_rollout_utils.go Outdated Show resolved Hide resolved
internal/providers/pluginfw/products/catalog/resource_function.go Outdated Show resolved Hide resolved
internal/providers/pluginfw/products/catalog/resource_function.go
Comment on lines +39 to +52
c.SetRequired("name")
c.SetRequired("catalog_name")
c.SetRequired("schema_name")
c.SetRequired("input_params")
c.SetRequired("data_type")
c.SetRequired("full_data_type")
c.SetRequired("routine_defintion")
c.SetRequired("is_deterministic")
c.SetRequired("is_null_call")
c.SetRequired("specific_name")
c.SetRequired("routine_body")
c.SetRequired("security_type")
c.SetRequired("sql_data_access")
c.SetRequired("parameter_style")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these actually needed? As in, if you don't have them, are they treated as optional?

Copy link
Contributor Author

@dgomez04 dgomez04 Dec 23, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I based myself on the REST API reference, did I interpret it correctly?

internal/providers/pluginfw/products/catalog/resource_function.go Outdated Show resolved Hide resolved
internal/providers/pluginfw/products/catalog/resource_function.go
c.SetReadOnly("updated_at")
c.SetReadOnly("updated_by")

return c
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked at UpdateFunction, and it seems like the only thing you can update about a function is its owner. Everything else cannot be updated. Can we mark all other fields with SetForceNew()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SetForceNew() doesn't seem to be a method of tfschema.CustomizableSchema. Am I looking at the wrong place? Any guidance here appreciate it.

@github-actions GitHub Actions
Copy link

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/terraform

Inputs:

  • PR number: 4189
  • Commit SHA: bb51da2a5609b4bf0639d27a041763a928ba8fa9

Checks will be approved automatically on success.

@dgomez04 dgomez04 requested a review from mgyucht December 23, 2024 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tanmay-db tanmay-db tanmay-db left review comments

@alexott alexott alexott requested changes

@hectorcast-db hectorcast-db Awaiting requested review from hectorcast-db hectorcast-db is a code owner automatically assigned from databricks/eng-plat-auto-exp-reviewers

@rauchy rauchy Awaiting requested review from rauchy

@mgyucht mgyucht Awaiting requested review from mgyucht

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEATURE] New feature request: Create function
6 participants
@dgomez04 @ian-norris-ncino @eng-dev-ecosystem-bot @alexott @mgyucht @tanmay-db