Community Wishlist Survey 2019/Multimedia and Commons/Musical notation – files on Commons (rendering and playback), graphical editing

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search
Random proposal ►◄ Multimedia and Commons  The survey has concluded. Here are the results!

Musical notation – files on Commons (rendering and playback), graphical editing

  • Problem: Although sheet music can already be embedded in articles to some extent, right now it's impossible to share digital sheet music as files on Commons. This limits the use of digital sheet music, which could otherwise be used to make synthesized audio files to compensate for the dearth of correctly-licensed recordings in Wikipedia articles and on other projects.

    US copyright has prevented almost all audio created in the 20th century from going into the public domain,[note 1] and will continue to be similarly restrictive for years to come; and using public domain sheet music doesn't easily convey to (most of) those reading it how something actually sounds. At the same time, the MediaWiki Score extension can already render scores and play MIDI files from LilyPond source code.[note 2] However, LilyPond lacks a graphical editor and its scores can't be used through the MediaWiki file syntax without exporting to PDF or OGG. The open-source notation program MuseScore is feature-rich and has a graphical interface, as well as a large and active community at musescore.com (Alexa ranking of 3,417[note 3]),[note 4] but its files cannot be rendered in MediaWiki at all. Conversion to LilyPond format requires a two-step conversion through the MusicXML format.[note 5] However, both the MuseScore application and the MuseScore command line app can export MuseScore files to various file formats which can already be rendered or played by MediaWiki.[note 6]

    Supporting MusicXML/MuseScore files on Commons would also potentially encourage MuseScore users to also upload scores to Commons,[note 7] and Commons would be able to take advantage of the large amount of transcribed public-domain sheet music that has already been uploaded to Musescore.com, including the verified and reviewed transcriptions that are being created as part of the OpenScore transcription project.[note 8] Because of Commons's more useful file categorization system and more stringent copyright enforcement, if Commons becomes another hub for MuseScore users it would also become easier for people to find digital sheet music and verify that a given score is in fact reusable for their purposes.

  • Who would benefit: Wikipedia readers, Wikisource readers and editors, Commons, LilyPond users, and the MuseScore community
  • Proposed solution: It would be beneficial for both readers and editors to allow notation files from Commons to be rendered and played.[note 9] This could be done by using the existing Score extension to render and synthesize the files,[note 10] by creating a new MediaWiki extension to allow them to be rendered separately,[note 11] by modifying the Media Viewer extension, or through a combination of those methods. There is an ongoing RfC at Commons to determine which file formats would be supported by Commons.[note 12][note 13]

    Another improvement that could be considered would be to integrate a graphical notation editor into VisualEditor, since this would enable editors to create scores graphically without having to worry about file conversion issues.[note 14] This would probably require writing a graphical interface for LilyPond or improving an existing unofficial one.

Notes

  1. ↑ Almost all audio created between 1923 and 1972 has a much longer copyright term than other works in the US due to being covered by state copyright law rather than federal copyright law until 2018, and the last recordings will only enter the public domain in 2067. See c:Commons:Hirtle chart#Sound recordings and w:en:Music Modernization Act.
  2. ↑ LilyPond can play both MIDI files generated from the score itself and MIDI files uploaded to Commons.
  3. ↑ As of 20 November 2018. This means that Alexa Internet measured it to be the 3,417th-most-popular website over the previous three months.
  4. ↑ Musescore.com is possibly the most active community for uploading sheet music, although this is in no small part due to much of its content being arrangements and transcriptions of copyrighted works.
  5. ↑ MuseScore files (.mscz) can be converted to MusicXML files (.mxl) in the MuseScore application, and vice versa; the open-source application Frescobaldi can convert MusicXML files to LilyPond files (.ly), but cannot do the reverse (although there are some conversion scripts around the internet). Most proprietary scorewriters, like Finale and Sibelius, can convert their files to the MusicXML format (or if not, there is usually a plugin or other program which can do that). None of these formats can be uploaded to Commons.
  6. ↑ The ones most useful for Commons would be SVG, PNG, OGG and PDF.
  7. ↑ Musescore.com has its own share of issues: users cannot update scores uploaded by others; its online player is unfortunately closed-source, unlike the MuseScore Windows/Mac/Linux application; and individual users can only upload five scores if they do not purchase a MuseScore Pro subscription. See the talk page for more background information.
  8. ↑ It might also be possible to benefit from the score validation software that is being developed for that project.
  9. ↑ Originally this proposal also asked for video or otherwise visual playback of scores. In the interest of more clearly indicating the goals of the request I have omitted that part.
  10. ↑ This would probably be accomplished by modifying the file syntax to allow specifying whether a notation file is rendered as PDF, as audio, or both.
  11. ↑ This would probably use the MuseScore app's command line to export files to PDF and OGG. It would be difficult to re-implement the entire MuseScore renderer, because the MuseScore format does not have a formal specification and is defined solely by the functionality of the application.
  12. ↑ While it could be sufficient to only allow MusicXML uploads, it may be desirable to additionally allow MuseScore files to be uploaded. This is because MuseScore's import and export of MusicXML is not perfect, and if two users use MuseScore to contribute to the same file then it may be beneficial for them to avoid importing and exporting to edit the MuseScore source file.
  13. ↑ Even if Community Tech doesn't have time to action this proposal, simply allowing uploads would allow musical notation files to be shared much more easily. For example, I've uploaded MuseScore exports to Commons before (such as File:Für Elise preview.svg), but have been unable to also upload the source file.
  14. ↑ Since this would be LilyPond-based, presumably the original LilyPond and its users would also benefit from the new graphical editor's existence.
  • More comments:
  • Phabricator tickets:
    • phab:T208494 – Allow music scores to be uploaded to Wikimedia Commons (1 November 2018)
    • phab:T201637 – Support engraving of musical files on Wikimedia Commons (10 August 2018)
    • phab:T209695 – Playback for musical notation files hosted on Wikimedia Commons (16 November 2018)
    • phab:T183642 – Support MusicXML to Lilypond Conversion (24 December 2017)
    • phab:T49528 – Create a VisualEditor plugin tool to add/edit musical scores (23 April 2013)

Discussion

Extended discussion on changing the content of the proposal. Jc86035 (talk) 11:37, 17 November 2018 (UTC)Reply[reply]
  • Phabricator tasks, as this touches upon many:
  • I have no strong opinion on whether MuseScore files should be upload-able to the commons:, but it should not detract from the efforts being made for the usability of the music rendering that already exists in MediaWiki. The music notation code from any program should be paste-able in the <score tag, and that is achievable through the acceptance of MusicXML. Having phab:T49528 (extend VisualEditor for Score) as the Wishlist task would be the most meaningful addition to the extension (and that is too big for me by myself).
I am however concerned with the native MuseScore format. Without any formal specification and how it's prone to corruption and incompatibilities with anterior versions, it carries a big caution against using it anywhere else than natively supported (musescore.org/.com).
The size limit for LilyPond scores is basically the size limit of the Wikipedia page itself. The best solution for what is presented here is two-fold; support MusicXML (which would make import from a multitude of score-writers easier) and implement creating scores in VisualEditor. Supporting MuseScore is not a true solution to our problems. Ebe123 (Communication | Activity report) 18:23, 30 October 2018 (UTC)Reply[reply]
@Ebe123: I have read that the MuseScore format is set to change again for 3.0, which is probably going to be released in early 2019. I agree it's problematic, but to me, the MuseScore end user, it outwardly appears that MuseScore is basically the only game in town because of its much larger community, which is why I suggested it over the LilyPond format, MusicXML or another format in the first place. (To others, noting that Ebe123 is also a LilyPond developer. I have nothing against him, of course, but it seems pertinent to note potential conflicts of interest, even if this isn't Wikipedia.)
I wouldn't mind if MusicXML was supported instead of the MuseScore format, but I've mostly only worked with MuseScore so it is more convenient for me personally to use MuseScore files, and some data (e.g. some text formatting, instrument "channel" switches which control audio output; visibility of staves) is lost when converting to MusicXML, although it's not clear to me whether this is an issue with MuseScore 2 or an issue with the standard itself. Jc86035 (talk) 18:49, 30 October 2018 (UTC)Reply[reply]
I may be missing something here, but if a MuseScore GPL command-line app can convert MuseScore files to formats which can be uploaded to commons, why not just immediately auto-convert any MuseScore files uploaded? It seems that that would avoids problems with format changes. A see-as-it-plays viewer with a cursor would be great. I think Cmglee does such things in pure HTML, so it might be possible. HLHJ (talk) 00:45, 31 October 2018 (UTC)Reply[reply]
@HLHJ: (Are you saying the files should be converted to PDF/OGG on upload, or they should be displayed as PDF/OGG?) The main purpose of allowing uploads of .mxl or .mscz files is that it would then be possible for other users to modify them as they please without having to transcribe the whole score again. The rendering/playback part of the proposal above is to internally convert the file to PDF and OGG (or, as proposed by Ebe123, to convert the whole file to LilyPond format and display it with the Score extension) and then display the file as both score and audio when the file syntax is used (or when it's used inside a <score> tag). Jc86035 (talk) 05:35, 31 October 2018 (UTC)Reply[reply]
Converting it to Lilypond would seem to make more sense. MuseScore seems a bit proprietary for Commons; I'd also prefer a Visual Editor interface for Lilypond if it's the user interface that's the problem. HLHJ (talk) 05:33, 8 November 2018 (UTC)Reply[reply]
Ouch, having a see-as-it-plays is hard even with JavaScript! It would also take lots of backend changes for things such as note spacings and tempo. Can't imagine anyone making one in plain HTML, nor would it be functional. Ebe123 (Communication | Activity report) 15:09, 31 October 2018 (UTC)Reply[reply]
I'll take your word for it, Ebe123, I am not that familiar with Lilypond or HTML 5. How about implementing the cursor as a separate overlay? Could that be done with an automatically-generated SVG animation?
MIDI instruction tags are included in the MusicXML specification. Ebe123 (Communication | Activity report) 15:09, 31 October 2018 (UTC)Reply[reply]
@Ebe123: I guess this might be a MuseScore bug, then? I might post on the MuseScore forum about this at some point, since the developers usually reply to posts and might know what's happening. Jc86035 (talk) 19:25, 31 October 2018 (UTC)Reply[reply]
I've created a forum post. Jc86035 (talk) 19:04, 1 November 2018 (UTC)Reply[reply]
@Ebe123: By "corruption and incompatibilities with anterior versions", did you mean the errors when opening older (1.x and 0.x) scores in MuseScore 2, or something else? Jc86035 (talk) 08:26, 31 October 2018 (UTC)Reply[reply]
Yes, and the fact that the files are easily corruptible (and that legal modifications are sometimes detected as corruption). Further because of the lack of a clear standard, there are incompatibilities within MuseScore within a version. Ebe123 (Communication | Activity report) 15:09, 31 October 2018 (UTC)Reply[reply]

To bring back on track, I don't think this request is really about file support. The sentiment behind it is that LilyPond has a huge learning curve, and so we should make it easier to write music. This should be accomplished not by implementing a new program, but prioritizing graphical music notation to the VisualEditor team and Score extension maintainer (me). Ebe123 (Communication | Activity report) 15:09, 31 October 2018 (UTC)Reply[reply]

@Ebe123: I think I agree with your characterization of this proposal, but I'm not totally sure. I agree that a lot more of this request is "allow .mxl/.mscz uploads" than it is "render .mxl/.mscz uploads", although the original point of asking for all of that together was that since the MuseScore command line can already export SVG and PDF it would hopefully not be particularly technically difficult to turn that functionality into a new extension (and using the MuseScore exporter itself would probably avoid some of the aforementioned issues).
(The main purposes of my requesting Commons uploads, to elaborate, are that without Commons uploading, because Wikipedia pages don't usually include the full text of works, full scores would almost certainly be relegated to Wikisource; it would be more difficult to use the score's audio on a page without the full score's code; and it would become more difficult to share/export/download scores.)
I generally agree with your concerns about MuseScore (perhaps particularly since it has been recently acquired, by Ultimate Guitar, and much of the development work is now being done by employees of that company). As the unaware end user, I can't really tell how much the issues matter, although they would definitely make developing an extension more difficult.
Would the VisualEditor integration mean that you/Community Tech/the VE team would be developing a full GUI for LilyPond, and then turning it into part of VisualEditor? Jc86035 (talk) 19:25, 31 October 2018 (UTC)Reply[reply]

@Jc86035: The WMF is generally supportive of allowing and facilitating hosting of any file formats that the community is interested in (as long as they don't have serious security problems). Indeed, we generally leave the decisions about which file formats to accept to the communities themselves. In this case, there has never been a discussion on Commons (that I'm aware of) about whether the community would like to host MuseScore or MusicXML files. If it turns out the community objects for some reason, it would be pointless for the WMF to invest work in supporting those formats. Would you be willing to go ahead and start an RfC on Commons asking whether or not the community would like to host either or both of those file formats? Sorry I didn't suggest that earlier, but potential voters will want to know if there is community support before backing this, and the WMF will want to know before agreeing to do any work around this. Ryan Kaldari (WMF) (talk) 21:31, 15 November 2018 (UTC)Reply[reply]

  • @Jc86035: Also, this proposal covers too many topics and Phabricator tickets. Please narrow the focus to whatever improvements would be the most important to you. Ryan Kaldari (WMF) (talk) 00:04, 16 November 2018 (UTC)Reply[reply]
    @Ryan Kaldari (WMF): I have begun an RfC. I've omitted some of the tasks since it's not really necessary for Community Tech to do them; hopefully this is more workable. Jc86035 (talk) 11:13, 16 November 2018 (UTC)Reply[reply]

User:Facenapalm User:Dispenser User:Ruslik0 User:Micru User:Hiàn User:Trockennasenaffe User:JAn Dudík User:Gryllida User:Tacsipacsi User:Superchilum User:Theklan User:M11rtinb User:MichaelSchoenitzer User:TheDJ User:Ayack User:Shizhao User:Thomas Obermair 4 User:Sannita User:Draceane User:Consulnico User:Liuxinyu970226 User:David1010 User:Mahir256 User:NMaia Notifying everyone who voted in the similar proposal from last year. (This proposal is, broadly, more precise and more feasible.) Jc86035 (talk) 15:05, 17 November 2018 (UTC)Reply[reply]

  • Comment Comment "However, LilyPond lacks a graphical editor and its scores can't be used through the MediaWiki file syntax without exporting to PDF or OGG." Misleading. Its scores can be inputted directly into MediaWiki. File syntax is not necessary and is to be avoided. Further, any notation would have to be converted into an image or audio for display on the web... The Score extension is also about to change (gerrit:370209) to use SVG and MP3 is supported.
"However, both the MuseScore application and the MuseScore command line app can export MuseScore files to various file formats which can already be rendered or played by MediaWiki." More conversions? Also I see MuseScore renderings to be of lesser quality and not as feature-rich as LilyPond (and they removed the LY exporter!). Its GUI is its "attractor".
Supporting MusicXML in the extension is basically a "beginner" task :) I also think the MuseScore format is very poor and so we should focus on improving the LilyPond experience, although I can see why allowing MuseScore files may be wanted. Ebe123 (Communication | Activity report) 02:18, 19 November 2018 (UTC)Reply[reply]
@Ebe123: Sorry, I seem to have written that sentence with the assumption that the reader knows that the Score extension supports LilyPond. Will fix. Quality is somewhat subjective, although LilyPond is unambiguously more customizable. Jc86035 (1) (talk) 16:26, 20 November 2018 (UTC)Reply[reply]

HLJH wrote:

“online WYSIWYG player/editor seems to be closed-source and freemium”: They use an OSS renderer internally on some places.

I’m also pretty sure that, if Wikimedia were to approach MuseScore/Ultimate Guitar, a Free solution could be found for code reuse suitable enough to make it usable here. (An online editor, though, is likely out of scope.) The profit model is to sustain development. Wikimedia adoption would rather drive more people to use it, so I’d think that would be more welcome. The MuseScore developers are very approachable and like the Free Software scene.

“I assume that it is also higher-bandwidth than the markup alternatives”: no. MuseScore files come in two flavours. *.mscx and *.mscz. The latter is the default, and it’s a PKZIP container with the mscx in it as well as any images etc. embedded into the score. The mscx is XML and comparable to MusicXML, but in my personal experience, MusicXML is more verbose. (MusicXML also has a compressed container format, *.mxl, but that’s rarely used.) I keep all my own scores as *.mscx in git and just batch-convert to other formats occasionally, and have written an MML to MusicXML exporter, so I have experience with both formats.

Mirabilos (talk) 23:37, 20 November 2018 (UTC)Reply[reply]

Thank you for the information and the talkpage ping, Mirabilos Full version of this reply at the RfC. It looks to me as if the trade-offs run:
Format Advantages Disadvantages Technical difficulty Lacks
MusicXML *common interchange standard *being replaced by MNX *can be imported by ~any player, designed as an interchange format, not for display *ongoing support
MuseScore *large community *online/mobile freemium elements *online visual editor and playback software exists, including mobile version, but is proprietary; would have to either duplicate and compete, or partly rely on third-party freemium service *copyleft online player/editor
LilyPond *already integrated into Mediawiki, fits markup orientation, unicode *does not natively export to MusicXML, only imports *open-source visual editor and playback software (not web-based) exists, but see phab:T49528 (mobile might be harder) *MusicXML or MNX export (NoteEdit, Rosegarden, and Frescobaldi, all GPL, have code for this)
MNX *should be perfect in every way :) *does not yet exist *starting from scratch (can be an advantage) *a full specification and an implementation
MEI/Verovio *browser-based, fast, wide and growing range of music cultures (including historic ones) encodable *does not do really sophisticated engraving (making pretty scores for print) *should be easiest, as open-source software exists, browser-based, SVG output (not sure if it does visual editing or just rapid previews) *non-academic users
I see that MuseScore is heavily involved in MNX development;[1] more MEI-focussed groups, including LilyPond, are not. MuseScore is also ably represented here on the Mediawikis by Peter Jonas (shoogle). Perhaps someone might also contact the LilyPond devs and MEI devs ask them to participate in this discussion, so we can get a broad range of viewpoints? HLHJ (talk) 04:06, 23 November 2018 (UTC)Reply[reply]
This discussion is continuing at the RfC, with useful comments from format developers. Comments from Mediawiki developers and other community members are also welcome. HLHJ (talk) 02:47, 29 November 2018 (UTC)Reply[reply]

Voting

Retrieved from "https://meta.wikimedia.org/w/index.php?title=Community_Wishlist_Survey_2019/Multimedia_and_Commons/Musical_notation_–_files_on_Commons_(rendering_and_playback),_graphical_editing&oldid=18665689"