Update index.html

AMAAI-Lab · Nov 16, 2023 · 77f2d01 · 77f2d01
1 parent 0ad678c
commit 77f2d01
Showing 1 changed file with 11 additions and 7 deletions.
diff --git a/index.html b/index.html
@@ -74,8 +74,10 @@ <h1 itemprop="headline" align="center">
             <br>
             <div class="container">
               <center>
-                <p><a href="https://arxiv.org/abs/2311.08355">[Paper on ArXiv]</a>&nbsp;&nbsp;&nbsp;&nbsp;<a href="https://github.com/AMAAI-Lab/mustango">[Code on GitHub]</a></p>
+                <p><a href="https://arxiv.org/abs/2311.08355">[Paper on ArXiv]</a>&nbsp;&nbsp;&nbsp;&nbsp;<a href="https://github.com/AMAAI-Lab/mustango">[Code on GitHub]</a>&nbsp;&nbsp;&nbsp;&nbsp;<a href="https://replicate.com/declare-lab/mustango">[Live demo]</a>&nbsp;&nbsp;&nbsp;&nbsp;<a href="https://huggingface.co/datasets/amaai-lab/MusicBench">[MusicBench dataset]</a></p>
               </center>
+
+
             </div>
             <h2 id="abstract">
               <font color="000093">Abstract</font>
@@ -84,6 +86,9 @@ <h2 id="abstract">
               <font color="061E61"> With recent advancements in text-to-audio and text-to-music based on latent diffusion models, the quality of generated content has been reaching new heights. The controllability of musical aspects, however, has not been explicitly explored in text-to-music systems yet. In this paper, we present Mustango, a music-domain-knowledge-inspired text-to-music system based on diffusion, that expands the Tango text-to-audio model. Mustango aims to control the generated music, not only with general text captions, but from more rich captions that could include specific instructions related to chords, beats, tempo, and key. As part of Mustango, we propose MuNet, a Music-Domain-Knowledge-Informed UNet sub-module to integrate these music-specific features, which we predict from the text prompt, as well as the general text embedding, into the diffusion denoising process. To overcome the limited availability of open datasets of music with text captions, we propose a novel data augmentation method that includes altering the harmonic, rhythmic, and dynamic aspects of music audio and using state-of-the-art Music Information Retrieval methods to extract the music features which will then be appended to the existing descriptions in text format. We release the resulting MusicBench dataset which contains over 52K instances and includes music-theory-based descriptions in the caption text. Through extensive experiments, we show that the quality of the music generated by Mustango is state-of-the-art, and the controllability through music-specific text prompts greatly outperforms other models in terms of desired chords, beat, key, and tempo, on multiple datasets. </font>
             </p>
 
+            <p style="color:#9F0909; font-style:italic"><b>Live demo available on <a href="https://replicate.com/declare-lab/mustango" style="color:#9F0909; font-style:italic">Replicate!</a></b></p>
+
+
             <h2 id="note">
               <font color="000093">Highlights</font>
             </h2>
@@ -108,7 +113,6 @@ <h2 id="note">
               </figcaption>
             </figure>
 
-
 </section>
 
 <br>
@@ -198,21 +202,21 @@ <h2><font color="061E61">Mustango generated samples:</font>
               Your browser does not support the audio element.
             </audio>
           </td>
-      </tr>
+      </tr> -->
     <tr>
         <td  class="mono">
             This is a live performance of a classical music piece. There is an orchestra performing the piece with a violin lead playing the main melody. The atmosphere is sentimental and heart-touching. This piece could be playing in the background at a classy restaurant. The chord progression in this song is Am7, Gm, Dm, A7, Dm. The beat is 3. This song is in Largo. The key of this song is D minor.        
         </td>
-    </tr> -->
+    </tr> 
 
-    <!-- <tr>
+     <tr>
           <td>
             <audio controls="controls">
               <source src="samples/samples2/output_116.wav" type="audio/wav">
               Your browser does not support the audio element.
             </audio>
           </td>
-    </tr> -->
+    </tr> 
 
     <tr>
         <td  class="mono">
@@ -594,7 +598,7 @@ <h3 id="Other comments">
             </h3>
             <div class="container">
               <p>
-                Our <a href="https://github.com/AMAAI-Lab/mustango">code and dataset</a> is available on github as open source. We thus aim to facilitate future music generation model development, training and evaluation. 
+                Our <a href="https://github.com/AMAAI-Lab/mustango">code and dataset</a> are available on github as open source. We thus aim to facilitate future music generation model development, training and evaluation. 
               </p>
             </div>
             <!-- <h3 id="Acknowledgement">