[Housekeeping] [flytekit] List differences on conflicting definition when registering

[wild-endeavor]

Describe the issue

Oftentimes, for a variety of reasons, users will see a stack trace like the following when running pyflyte run or registering a Flyte entity through FlyteRemote.

Traceback (most recent call last):
  File "/fn/bin/flyte-cli", line 8, in <module>
    sys.exit(_flyte_cli())
  File "/fn/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/fn/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/fn/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/fn/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/fn/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/fn/lib/python3.9/site-packages/flytekit/clis/flyte_cli/main.py", line 1775, in register_files
    _extract_and_register(client, project, domain, version, files, patches)
  File "/fn/lib/python3.9/site-packages/flytekit/clis/flyte_cli/main.py", line 1711, in _extract_and_register
    client.raw.create_workflow(_workflow_pb2.WorkflowCreateRequest(id=id, spec=flyte_entity))
  File "/fn/lib/python3.9/site-packages/flytekit/clients/raw.py", line 128, in handler
    return fn(*args, **kwargs)
  File "/fn/lib/python3.9/site-packages/flytekit/clients/raw.py", line 170, in handler
    raise e
  File "/fn/lib/python3.9/site-packages/flytekit/clients/raw.py", line 163, in handler
    fn(self, create_request)
  File "/fn/lib/python3.9/site-packages/flytekit/clients/raw.py", line 350, in create_workflow
    return self._stub.CreateWorkflow(workflow_create_request, metadata=self._metadata)
  File "/fn/lib/python3.9/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/fn/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.INVALID_ARGUMENT
	details = "workflow with different structure already exists with id resource_type:WORKFLOW project:"flyte-sandbox-jeevb" domain:"main" name:"sandbox.subworkflow.parent_workflow" version:"vrinda-spike-a3c802abfbd5" "
	debug_error_string = "{"created":"@1652812018.080534684","description":"Error received from peer ipv4:192.168.65.2:64766","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"workflow with different structure already exists with id resource_type:WORKFLOW project:"flyte-sandbox-jeevb" domain:"main" name:"sandbox.subworkflow.parent_workflow" version:"vrinda-spike-a3c802abfbd5" ","grpc_status":3}"

This is not very helpful however. Upon this error flytekit should fetch the conflicting resource from Admin, and compare the result, informing the user in the error message where the differences are.

Related component(s)

flytekit

This could also be done in Admin itself, and returned along with the message. Admin doesn't have more information though necessarily, since it's just comparing a hash. So a diff will have to be done regardless and it's probably easier to do in flytekit. Though doing it in Admin means that flytectl would get this for free as well.

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[jerempy]

I'm taking a whack at this one, just FYI

[jerempy]

@samhita-alla I got a couple PRs up for this one 😎

[holdsworth]

How to overcome this issue?

[jerempy]

@holdsworth not sure what problem you are facing. The issue here was about error messages in using pyflyte not providing enough information and the resolution of this issue sought to just provide more information to help the user in debugging. If you are having a problem you might get some help by either looking for a similar issue to the problem you are having, creating an issue, or joining the slack