Remove Grouper and LDAP from OpenConext

Introduction

Grouper and it's database backend LDAP have long been part of OpenConext. Since EngineBlock 5.0 it is no longer necessary to store the users in LDAP, and since Teams 6.0.0 it is no longer necessary to have grouper as a backend for group management. In order to phase out LDAP entirely from your OpenConext installation, several steps are needed, which are documented below. First, the method to migrate the LDAP users to the Engineblock database is shown. Then the way to migrate the users and groups from grouper to the Teanms database is described. The whole operation requires that the steps are followed exactly as shown.

Engineblock user migration:

Since Engineblock 5.0, a toggle has been introduced to enable or disable LDAP as a backend. You can find this toggle in /etc/openconext/engineblock.ini under this variable: engineblock.feature.ldap_integration

When set to 1, ldap as a backend is enabled. When set to 0, ldap is disabled. Please note that since Engineblock 5.0, when the toggle is set to "1", users are provisioned to both the LDAP and the local database. Migration is only needed to migrate the users from Engineblock 4.x and before.

If you started using Engineblock from version 5.0.0 or upwards, the only thing to do is to set engineblock.feature.ldap_integration to 0.

If you need to migrate users that have been provisioned from before engineblock 5.0, please follow these steps:

1. Download this ldap2csv script: https://gist.github.com/dansimau/674203
2. Execute the following command:

ldapsearch -LLL   "(&(ObjectClass=collabPerson)(CollabPersonUUID=*))"  -D "cn=admin,dc=_YOURORGANIZATION,dc=YOUR_TLD" -w $SECRET  -h YOUR_LDAP_HOST  -b "dc=YOURORGANIZATION,dc=YOUR_TLD" collabPersonId collabPersonUUID | ./ldif-to-csv.sh collabPersonId collabPersonUUID | sed s'/^/INSERT INTO USER(collab_person_id,uuid) VALUES(/g'  | sed s'/$/)/' > users_migrated_from_ldap.sql

Where: dc=_YOURORGANIZATION,dc=YOUR_TLD represents your LDAP suffix and LDAP_HOST represents the hostname of your ldap server.

4. The results is a list of all users that are in the ldap, in SQL format. Import those in your eb database:

mysql -u root -p eb < users_migrated_from_ldap.sql

5. Switch engineblock to not use LDAP anymore. Set engineblock.feature.ldap_integration to 0. Ansible can be used as well for this. Make sure that the variable engine_feature_ldap_integration is set to 0 in your group_vars and reprovision eb:

ansible-playbook -i /path/to/inventory your_playbook.yml --tags eb -u your_username -K -e secrets_file=/path/to/secrets.yml

6. If you are not doing the Teams migration at the same time, you should make sure that Teams provisions your users to LDAP. To do this, the /opt/teams/application.properties shuld contain this line: provision.users=true

The corresponding group variable is teams_provision_users. Set it to true in your group_vars and reprovision Teams:

ansible-playbook -i /path/to/inventory your_playbook.yml --tags teams -u your_username -K -e secrets_file=/path/to/secrets.yml

Phase out grouper and migrate users to Teams database.

This is somewhat more simple, and requires to use specific tags in OpenConext-deploy.

After these steps are done, the grouper groups and users are migrated to the Teams database. Teams will use it's own database as backend, and voot queries Teams in stead of Grouper for group memberships.

1. Add these secrets to your secrets file:

teams_migration_secret_key: secret

Add a secrets file for the password for the teams GroupProvider:

external_group_provider_secrets:
 teams: secret

2. Install version 5.1.1 of Teams:

cd OpenConext-deploy
git pull
git checkout 227

Make sure that teams_version in your group_vars is set to 5.1.1:

teams_version: 5.1.1

Then provision teams:

ansible-playbook -i /path/to/inventory your_playbook.yml --tags teams -u your_username -K -e secrets_file=/path/to/secrets.yml

3. Migrate the users and groups

curl 'https://your_teams_url/migrate?key={teams_migration_secret_key}

Check /var/log/teams/teams.log for any errors.

If there are no errors, you can continue upgrading Teams and Voot. This can be any other moment. The only change to Teams 5.1.1 is that is is able to do the migrations.

4. Install Teams 6.0.7

cd OpenConext-deploy
git pull
git checkout 229

Make sure that teams_version in your group_vars is set to 6.0.7:

teams_version: 6.0.7

Then provision teams:

ansible-playbook -i /path/to/inventory your_playbook.yml --tags teams -u your_username -K -e secrets_file=/path/to/secrets.yml

Rerun the migration:

curl 'https://your_teams_url/migrate?key={teams_migration_secret_key}

5. Install voot 3.0.0

First change your group_vars voot.externalGroupProviders. Remove the grouper entry, and add teams, like this:

externalGroupProviders:
    - {
        type: "teams",
        url: "https://teams.{{ base_domain }}/api/voot",
        credentials: {
         username: "{{ teams.voot_api_user }}",
         secret: "{{ external_group_provider_secrets.teams }}"
        },
        schacHomeOrganization: "{{ base_domain}}",
        name: "SURFteams",
        timeoutMillis: 15000
}

ansible-playbook -i /path/to/inventory your_playbook.yml --tags voot -u your_username -K -e secrets_file=/path/to/secrets.yml

6. Test (eg by using authz-playground)

7. At a later time you can remove LDAP, grouper and Tomcat (provided you are not using OIDC)