Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Facilitate Custom File Formats - Required For Mastodon-Git Project #329

Merged
merged 2 commits into from
Oct 9, 2024

Conversation

maarzt
Copy link
Contributor

@maarzt maarzt commented Oct 8, 2024

The "Mastodon-Git" plugin aims to efficiently store Mastodon datasets inside a git repository. Unfortunately git performs very poorly when processing a history of large binary files as produced by the default Mastodon file format. Mastodon-Git therefor implements a customized Mastodon file format that can very efficiently be processed with git.

Two classes in the Mastodon core need to be public in order for mastodon-git to implement this custom file format that can still benefit from mastodon core features:

  • fast serialization of Spot and Link into byte arrays (class ModelSerializer)
  • and serialization of the GUI state into an XML (class MamutViewStateXMLSerialization)

(Mastodon-git currently has its own copies of the ProjectLoader and ProjectSaver classes. In the long term it would be amazing if we could add a plugin API into mastodon core. That supports the implementation of custom file formats. Or at least make the ProjectLoader and -Saver more reusable. However in the short term it would be great if the ModelSerializer and MamutViewStateXMLSerialization were public. This would allow us the further experiment with the mastodon-git file format, without the need to use a customized mastodon.jar file.)

maarzt added 2 commits October 8, 2024 23:11
This change allows other classes to write & read spots into "raw" format.
Being able to access this class is useful when implementing code
that serializes a Mastodon project into as custom file format. This change
is helpful for implementing a optimized file format in mastodon-git.
Making this class more accessible allows classes outside this package to
read and write "gui.xml" files. It is particularly useful for mastodon-git.
@maarzt maarzt changed the base branch from master to dev October 8, 2024 21:49
@tinevez
Copy link
Contributor

tinevez commented Oct 9, 2024

Hello. Let's do this.
Your comments are very helpful. Maybe we should keep track of them in a github project if we revise the IO.
I feel like Mastodon could be very useful as a viewer / editor for other tracking tools. A good extensible IO will be required if we are to go this way.

@tinevez tinevez merged commit fcabb52 into dev Oct 9, 2024
1 check failed
@tinevez tinevez deleted the facilitate-custom-file-formats branch October 9, 2024 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants