Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce support for Slack Enterprise Grid #47

Open
wants to merge 16 commits into
base: master
Choose a base branch
from

Conversation

coltoneshaw
Copy link
Member

@coltoneshaw coltoneshaw commented Jan 11, 2024

Summary

This PR introduces support for Slack Enterprise grid within mmetl.

How to use this PR

Simply provide it a slack grid export and an accompanying teams.json file.

./mmetl grid-transform -f slackexport.zip -t teams.json

Teams file example. T0001 is the team ID that is found within a post at post.Team. teamx is the name of the team, which corresponds to the folder name inside teams/.

{
  "T0001" : "team1",
  "T0002" : "team2",
}

What is Enterprise Grid?

It's a collection of slack workspaces under a single organization within Slack. They are able to share channels across slack workspaces and collaborate this way.

The files structure is like:

Enterprise Grid Export/
├── channel1/
│   ├── 2023-01-01.json
│   └── 2023-01-02.json
├── channel2/
│   ├── 2023-05-01.json
│   └── 2023-05-02.json
├── teams/
│   ├── team1/
│   │   ├── channel3/
│   │   │   ├── 2023-05-01.json
│   │   │   └── 2023-05-02.json
│   │   ├── channels.json
│   │   ├── mpims.json
│   │   ├── dms.json
│   │   ├── users.json
│   │   └── groups.json
│   └── team2/
│       ├── channel4/
│       │   ├── 2023-05-01.json
│       │   └── 2023-05-02.json
│       ├── channels.json
│       ├── mpims.json
│       ├── dms.json
│       ├── users.json
│       └── groups.json
├── channels.json
├── org_users.json
├── mpims.json
├── dms.json
└── groups.json

What does this do?

This PR simply processes the enterprise grid export file by parsing the root level channels for the appropriate team, and placing them there. Then when done, zipping up every team folder as a separate slack export.

How do we know where the channels belong?

For the most part, every channel has a post that has a Team attribute on it, like below. These team attributes are used to identify the originating team from the channel and move it there. The appropriate .json file is then updated with the new information. If for whatever reason a matching team is not found, that item is skipped on the import entirely.

    {
        "client_msg_id": "",
        "type": "",
        "text": "",
        "user": "U1",
        "ts": "1695219722.430309",
        "blocks": [  ],
        "team": "team1",
        "user_team": "team1",
        "source_team": "team1",
        "user_profile": { }
    },

Potential Gaps

One potential gap is that this does not go down the list of channel members and repopulate the users.json with the members from the org_users.json file. The reason is they will most likely be imported multiple teams into their space anyways, so those users would be filled out and added properly. However, it wouldn't be a difficult thing to add here.

@hanzei hanzei added the 2: Dev Review Requires review by a core committer label Jan 23, 2024
Copy link
Contributor

@mickmister mickmister left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just some comments for discussion. Nice work!

services/slack_grid/precheck.go Outdated Show resolved Hide resolved
services/slack_grid/parse.go Outdated Show resolved Hide resolved
services/slack_grid/parse.go Outdated Show resolved Hide resolved
services/slack_grid/parse.go Outdated Show resolved Hide resolved
services/slack_grid/parse.go Outdated Show resolved Hide resolved
@coltoneshaw coltoneshaw requested a review from mickmister March 11, 2024 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2: Dev Review Requires review by a core committer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants