Export to MusicXML

[daveyarwood]

Moved from #44.

MusicXML is a format that a lot of GUI score editing programs can import/export from/to. Being able to import/export between MusicXML and Alda scores would be valuable because it would allow us to hook into that ecosystem.

This may also open the door to import/export from/to MIDI, with MusicXML as an intermediate step.

[iggar]

Hello, I was just looking into this and wondering how this could be implemented:
a) a standalone command line tool that would convert/translate alda scores to MusicXML, something like alda2musicxml <filename>
b) new output to the parse command, something like alda parse -f myscore.alda -o musicxml.

[daveyarwood]

I think both ideas have merit, but I like the idea of the alda release executable being a batteries-included program that can do it all.

I'm imagining a new alda export command, which will eventually be able to export in a variety of formats, including MIDI, LilyPond, wav/mp3, etc. Usage might look something like alda export -i myscore.alda -o myscore.musicxml -t musicxml, with the -t/--type flag being optional -- when left out, we can infer the type based on the file extension of the -o/--output argument.

Another idea is to use Unix input/output conventions, and read from STDIN, write to STDOUT, something like cat myscore.alda | alda export -t musicxml > myscore.musicxml. It seems like we ought to be able to support doing it both ways, i.e. if you omit the -i/--input flag, we read from STDIN, and if you omit the -o/--output flag, we write to STDOUT instead of to a file.

[daveyarwood]

Open question: maybe the command should be called alda convert, given that it's both importing and exporting?

[TBuc]

PLEASE IGNORE THIS COMMENT - See two answers below.

I'd say to keep import and export separated since:

• import and export are really clear and sound commands
• there aren't so many options (just two, import and export), so merging them into an upper-level command wouldn't shorten the commands list much
• convert makes me think of some output file, which seems not to be always the case, while giving uncertainty about the input (will it require a file? or a score?). Yes, reading the help would suffice, but why should I do it when I can have two sound commands that don't require me to investigate at this level, and just read the help for the final syntax rather for than the purpose first?

[daveyarwood]

In general, I do like things to be modular, so having both import and export would be nice in that way.

But, the semantics of these hypothetical commands are not clear to me.

What is the outcome of alda import -f somefile.mid?

If alda export is a separate command that does not involve importing, then what does it export?

[TBuc]

I totally misunderstood the context - thought we were in the CLI landscape. Sorry guys, please ignore my previous comment.

My opinion is that convert is, as you propose, the best choice.

[brentjanderson]

I have zero experience with Alda or MusicXML, but one suggestion I'd throw into the mix is tapping into Pandoc for the conversion pipeline. It has the facilities built for doing a lot of this work, so a custom alda converter for pandoc could target any number of formats. Someone with lua experience might be able to develop a custom pandoc converter, and this would open up a lot of potential.

[jaredforth]

@daveyarwood

Has there been any progress on this item? I've been exploring Alda this week and want to contribute to the project. This is something that I am interested in tackling, but it doesn't seem like there is necessarily a road map for how this feature will be implemented as of yet.

[daveyarwood]

Hi @jaredforth! There has not been progress on this to my knowledge. I'd love to see this feature move forward, and I would be happy to help provide direction, spec things out, answer questions, etc.

After re-reading the discussion above, I still like the direction of the last thing I proposed, except I've totally changed my mind about import and export not making sense. I think import and export have clear semantics, and it would be more intuitive to have import and export vs. a single convert command.

Another important development is that since we last discussed this feature, we did, in fact, add an alda export command, so that also makes me more inclined to choose import and export.

Commands:

• alda export takes some Alda input (either via -c, --code, -f, --file, or piped into stdin) and writes output in another format

  • Currently we only support MIDI; this issue is about adding MusicXML as an output format.
  • Currently the output is written to a file whose filename you have to specify. I think it would be nice and Unixy if we also supported printing the output to stdout so that you can pipe it into a file or maybe into another process.

• alda import (not yet implemented) takes some input in another format and writes Alda source code

  • Options would be very similar to alda export, including the input format and the desired output filename (default: just write the output to stdout)

Here are some hypothetical usage examples:

#### Export ####

# These work already (see `alda export -h`)
alda export -c "organ: [ c16 d e f ]*3 g2" -o /tmp/organ.mid
alda export -f my-score.alda -o my-score.mid
echo "glockenspiel: o5 g8 < g > g e4 d4." | alda export -o /tmp/glock.mid

# New feature: if you leave off the file name, it prints to stdout
alda export -c "piano: c8 d e f g" -F midi > /tmp/piano.mid
alda export -f some-score.alda -F midi > some-score.mid
echo "piano: c8 d e f g" -F | alda export -F midi > /tmp/piano.mid

# New feature: musicxml format option
alda export -c "tuba: o2 c1~1~1" -F musicxml > tuba.musicxml
# If you don't specify a format, we can infer it from the file extension
alda export -c "tuba: o2 c1~1~1" -o tuba.musicxml

#### Import ####

# The input format (musicxml in this case) can be inferred from the file
# extension.
alda import -f my-score.musicxml -o my-score.alda
# If the input filename isn't available, the `-F, --format` option is required
# so that Alda knows how to parse the input.
alda import -c '<?xml version="1.0" ... etc. ...' -F musicxml -o my-score.alda
cat my-score.musicxml | alda import -F musicxml > my-score.alda

# Importing MIDI files would also be super fun!
alda import -f gangnam-style.mid > gangnam-style.alda # edit, explore, tweak

### Living the Unix dream ###

# Using `alda play` as a MusicXML jukebox
alda import -f something.musicxml | alda play
# Using `alda play` as a MIDI file jukebox
alda import -f something.mid | alda play
# Using Alda as an intermediate representation to convert MusicXML to MIDI and
# vice versa
alda import -f something.musicxml | alda export -F midi
alda import -f something.mid | alda export -F musicxml
# Imagine if round-tripping through several formats worked. How crazy would that
# be?
cat "bassoon: (quant 30) c16*4 (quant 100) c4" \
  | alda export -F musicxml \
  | alda import -F musicxml \
  | alda export -F midi \
  | alda import -F midi \
  | alda play

Supporting import and supporting export are both complicated endeavours, and there's no reason that we absolutely have to have both at the same time. Both present their own challenges. Importing from MusicXML is challenging because we have to figure out how to support the features of MusicXML in Alda. Whereas exporting to MusicXML is challenging in a different way because we have to figure out how to support the features of Alda in MusicXML.

Another thing worth mentioning at this point is that I am now very far along in rewriting Alda from the ground up (see also the v2 branch of alda-lang/alda). All of the core logic is in the client now (written in Go) and it's stable enough at this point that I don't think it makes sense to continue to develop any new features in Java/Clojure for Alda v1. I'd be happy to help get you up to speed with contributing to the v2 branch if it's confusing at all; I've been pretty diligent about documenting stuff in the README, so hopefully it is easy enough to check out the v2 branch and try it out 🤞

By the way, the #development channel in the Alda Slack group is a good place for guidance on implementation, if you'd prefer that over discussing here in the GitHub issue.

[jaredforth]

@daveyarwood

I like the import and export semantics as well, and I will certainly check out the v2 branch! I have significantly more experience in Go than Closure so that makes contribution easier for me. I just joined the Slack group, so we can definitely have further discussion there.

[daveyarwood]

For our reference: a new blog post from @infojunkie about validating MusicXML: https://blog.karimratib.me/2020/11/17/validate-musicxml.html

[yasz]

Hello, What is the progress now?

[daveyarwood]

@Scowluga has done some great work beginning to implement the basic functionality of importing MusicXML scores and generating working Alda code. There is still a ways to go, because the more we've been learning about this, the more complex we are finding this to be! :) But I think we have a good overall plan, and it's just a matter of finding the time to work on it.

At the moment, I think we're waiting on me to do some refactoring to make this work easier - specifically, to have the Alda parser return an actual AST at one point. I'm currently busy preparing for my Alda workshop at Strange Loop at the end of the month, but this refactor is on my TODO list!

[daveyarwood]

I finished the aforementioned refactor over my holiday break and it's on the master branch. I don't think there is anybody currently working on this, so if anyone is interested in picking up the torch from @Scowluga in implementing this feature, let me know and I can fill you in on the current status and point you in the right direction!

[suntong]

That's great.

So I took a look at the source, and found

• JSON returns a JSON representation of an ASTNode
• HumanReadableAST returns a human-readable textual representation of an AST.

Are these two functions exposed and accessible from cli?

[daveyarwood]

@suntong Sorry I missed this! Yes, this functionality is exposed via the CLI. See example usage here.

[ManHinnn0509]

Hi, I assume this function is not finished yet and the export can only export as midi file?

[daveyarwood]

Yes, that's right.

[daveyarwood]

I am not actively focused on this, so if anyone reading this would like to work on it, let me know and I'll be happy to help point you in the right direction!

[truj]

Yes, that's correct.
see here