Sharepoint Online: sync not working on new tenant due to Azure ACS retirement

[jlnmrtn]

Bug Description

Sync not working for the sharepoint online connector for newly created (after 01/11/24) MS365 tenants, because ACS got retired for new tenant since November 1st 2024 as per: https://learn.microsoft.com/en-us/sharepoint/dev/sp-add-ins/retirement-announcement-for-azure-acs

To Reproduce

Steps to reproduce the behavior:

1. Create a new MS365 tenant
2. Create a sharepoint site
3. Follow the documention https://www.elastic.co/guide/en/elasticsearch/reference/current/es-connectors-sharepoint-online.html#es-connectors-sharepoi[…]-oauth-app-permissions
4. You will hit the first error, indeed unable to turn ACS on again using the powershell cmdlet
5. Second issue you will get is by running the connector:

aiohttp.client_exceptions.ClientResponseError: 401, message='Unauthorized', url="https://ttj57.sharepoint.com/sites/scoobypoint/_api/web/lists/GetByTitle('Site%20Pages')/items?$select=*,EncodedAbsUrl"

Expected behavior

No 401 authorization error , sync happening correctly

Environment

• Elasticsearch 8.16.1
• Connector 8.16.1 , managed and selfmanaged

Additional context

Permissions issue can be reproduce by doing the following call (an access token need to be obtained first):

• Request:

curl --location --globoff 'https://ttj57.sharepoint.com/_api/web/lists/GetByTitle('\''{list_title}'\'')' \
--header 'Authorization: Bearer 1NiIsIng1dC....' \
--header 'Accept: application/json;odata=verbose'

• Answer:

{
    "error": "invalid_request",
    "error_description": "Token type is not allowed."
}

This error is due to ACS disabled

[seanstory]

@danajuratoni @erikcurrin-elastic I'm leaving this as "high" for now, since it does seem like a significant regression. But also the most urgent deadline for fixing this (to prevent breakage for existing users) is April 2026, so there's some grey area.

[artem-shelkovnikov]

Connected thread: https://elastic.slack.com/archives/C07E6J58STD/p1733741133497399

I took some time to research and so far the following findings are:

1. We use REST client for several things: fetching site pages, fetching list item attachments + fetch user permissions for site pages and objects
2. Site pages are already available via Graph API: https://devblogs.microsoft.com/microsoft365dev/microsoft-graph-api-for-sharepoint-pages-is-now-generally-available/
3. Other things are seemingly not very well available

Entra ID does not have a lot of documentation, so I will have a call with our fellow Elastician who was able to make it work to see how.