Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure Blob Storage, Azure File Share and SharePoint Online Connector Apps #23225

Open
wants to merge 40 commits into
base: main
Choose a base branch
from

Conversation

IceOnly
Copy link

@IceOnly IceOnly commented May 5, 2023

This PR contains three new connector apps, to connect Azure Blob Storage, Azure File Share and SharePoint Online with the New File System Module in the System App.

File System Module PR:
microsoft/BCApps#663

Here are some Screenshots:
image
image
image
image
image
image

@pri-kise
Copy link
Contributor

pri-kise commented May 5, 2023

Some general thoughts:
I think we need some kind of path serparated from the name added to the file account content.
It's often very important to know the path.

Additionally we might need some additional kind of top level selection before List Directories:
I'm thinking about a webservice for local file system access, there we should be able to select the Drive "C:" or "D:".
For Azure File Share we should be able to select the File Share on a top level before searching directories and files.

Further more if I take a look at the current sharepoint client. I have no idea how we should map this to the current provided
interface "File Connector".
https://github.com/microsoft/ALAppExtensions/blob/main/Modules/System/SharePoint/src/SharePointClient.Codeunit.al

Sometimes a path isn't enough and we might need more information to download a file.
Maybe this info could be added to the table 70005 "File Account Content" table and the interface "File Connector" could accept a "File Account Content" as parameter instead of a path....

Maybe someone else with more experience has some ideas how this access can be unified...

@IceOnly
Copy link
Author

IceOnly commented May 6, 2023

Some general thoughts: I think we need some kind of path serparated from the name added to the file account content. It's often very important to know the path.

Additionally we might need some additional kind of top level selection before List Directories: I'm thinking about a webservice for local file system access, there we should be able to select the Drive "C:" or "D:". For Azure File Share we should be able to select the File Share on a top level before searching directories and files.

Further more if I take a look at the current sharepoint client. I have no idea how we should map this to the current provided interface "File Connector". https://github.com/microsoft/ALAppExtensions/blob/main/Modules/System/SharePoint/src/SharePointClient.Codeunit.al

Sometimes a path isn't enough and we might need more information to download a file. Maybe this info could be added to the table 70005 "File Account Content" table and the interface "File Connector" could accept a "File Account Content" as parameter instead of a path....

Maybe someone else with more experience has some ideas how this access can be unified...

Why not listing C: and D: as directory on the top level?
The path is part of the Fiel Acount Content table. It is splitted in Parent Folder and Name.

However, I am not sure if the module should define the path separator. So all Services uses the same path structure and the implentation need to translate if needed.

Copy link
Contributor

@PeterConijn PeterConijn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks cool! I have a few small performance and security concerns, which I have addressed, and some minor other stuff, but nothing major. I'm excited for this.

@JesperSchulz
Copy link
Contributor

JesperSchulz commented Aug 18, 2023

Let me amplify this on Twitter/X to get your initial question answered: "before I go round the whole thing, I'm interested in whether there is any interest in the module at all". Let's see what people have to say! 😊

@tinfister
Copy link

I support every effort to improve file handling.

@dNsl9r
Copy link

dNsl9r commented Aug 18, 2023

Wonderful! Please merge this 👍🏻

@miljance
Copy link
Contributor

Easily overlooked feature but a needed one. Huge development effort that should not be thrown away.
Perhaps I will find some time to invest in contribution from my side but cannot make promises at the moment.

@TheDoubleH
Copy link

Bravo! Love seeing more 'helper' features/functions that eventually will benefit all of us!

@JesperSchulz
Copy link
Contributor

@IceOnly, this PR seems to have stagnated a little (which is perfectly fine). I was just wondering if you want to push this forward at some point, or if you've changed your mind / cannot find the time? There seems to be plenty of interest for this module in the community!

@IceOnly IceOnly requested a review from a team as a code owner October 6, 2023 07:26
@IceOnly IceOnly requested a review from dagirard October 6, 2023 07:26
@IceOnly
Copy link
Author

IceOnly commented Oct 6, 2023

@IceOnly, this PR seems to have stagnated a little (which is perfectly fine). I was just wondering if you want to push this forward at some point, or if you've changed your mind / cannot find the time? There seems to be plenty of interest for this module in the community!

It is the time. I will try to find some free time to make the last changes.

This are my open points:

  • I am struggling with the wist from pri-kise to split the Main Interface in logic interfaces like one for File, one for Directory and one for Account access. I like the Idea to implement one Interface and you are done.
  • I will try to implement a realy simple pagination interface. The Most file services uses realy different methods for getting the next batch. So i will only alow the implemntation to result a pagination codeunit that can store the inforamtion it needs to get the next batch. How big a batch is can only be defined by the implementation.

@JesperSchulz
Copy link
Contributor

Feel free to do it at your own pace. I didn't mean to stress you. I was just curious if you still intend to continue work. If you do, we'll just leave this PR open. No problem whatsoever.

@IceOnly IceOnly changed the title Azure Blob Storage Management App Azure Blob Storage, Azure File Share and SharePoint Online Connector Apps Mar 1, 2024
@IceOnly
Copy link
Author

IceOnly commented Mar 1, 2024

@JesperSchulz I need an three Object Ranges for this Apps. Which one can I used? The highest object type count is three objects.
I'm also not sure which namespace to use. Or simply use none, as in the email apps?


Caption = 'SharePoint Account';

fields
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@IceOnly could you maybe add the 2nd Authentication Type for SharePoint that was added recently .
This would require a few new Fields: Enum SharePointAuthType , a "Certificate Password Key" and a Certificate Password (Blob)

See: microsoft/BCApps#705

var
SharePointFile: Record "SharePoint File";
SharePointClient: Codeunit "SharePoint Client";
TempBlob, TempBlob2 : Codeunit "Temp Blob";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TempBlob2 is unused.

@IceOnly regarding your issue with the empty stream.
Take a look at the following issue: Stream

Why aren't you directly using the parameter Stream in the procedure DownloadFileContentByServerRelativeUrl?

@JesperSchulz JesperSchulz added the Integration GitHub request for Integration area label Apr 22, 2024
@JesperSchulz JesperSchulz self-assigned this Dec 6, 2024
JesperSchulz added a commit to microsoft/BCApps that referenced this pull request Dec 20, 2024
Migration of microsoft/ALAppExtensions#23225

Now that we have made several clients Universal code ready.
One of the main problems turned out to be the replacement of the file
record and the file object (File.Create, File.Exists).
Some of our customers now use Azure Blob Service from the System app.
Others don't have a good internet and now use a local microservice with
REST API, others use Azure File Shares.
The problem is that every integration is different. I think many other
partners will have the same problem.
To simplify access I have adopted the email module and created new file
accounts.
With just one codeunit, a developer can now connect to various file
services without having to know how they actually work. The code unit
"File System" delviers everything taht is needed.

An additional PR contains three connector apps:
microsoft/ALAppExtensions#23225

This Apps will conenct:
 - Azure Blob Storage
 - Azure File Share
 - SharePoint Online

All three service can be access over **one** unified interface!

New provider in the future could be:
 - a OneDrive Provider
 - Third Party file services

An example that shows how simple it is to use the new modules can be
found here:
https://github.com/IceOnly/BC_FileSystem_Example

But before I go round the whole thing, I'm interested in whether there
is any interest in the module at all.

Here are some Screenshots:

![image](https://user-images.githubusercontent.com/3911556/236433843-11ee0b26-ee9c-4da2-8bc6-efb32675090c.png)

![image](https://user-images.githubusercontent.com/3911556/236433887-bf2bf2f2-f68e-4efa-9db5-074a6f68fad0.png)

![image](https://user-images.githubusercontent.com/3911556/236433950-e8117496-16f6-4b25-a575-42ba729ca6c3.png)

![image](https://user-images.githubusercontent.com/3911556/236434182-97fd304a-95ac-4180-9d52-9ead78899822.png)

![image](https://user-images.githubusercontent.com/3911556/236435173-a01a77bc-04de-4063-b0bb-bf18e28b2817.png)

![image](https://user-images.githubusercontent.com/3911556/236435258-56d68d50-7581-4527-bcdd-5b6e3ed53ac8.png)

I have decided in favour of some restrictions. Paths must not start with
a /. The only supported path separator is /. If services that require a
\ are to be connected, this must be translated by the connector app.

These restrictions should ensure that the file service can be exchanged
without upgrading data.

Assigned Workitems:
Fixes #2418 
Fixes
[AB#559148](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/559148)

---------

Co-authored-by: Stefan Sosic <[email protected]>
Co-authored-by: Jesper Schulz-Wedde <[email protected]>
Co-authored-by: Darrick <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Integration GitHub request for Integration area
Projects
None yet
Development

Successfully merging this pull request may close these issues.