-
-
Notifications
You must be signed in to change notification settings - Fork 264
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update distube dependencies for improved YouTube support #1180
base: master
Are you sure you want to change the base?
Conversation
📦 🤖 A new release has been made for this pull request.To play around with this PR, pull Images are available for x86_64 and ARM64.
|
@timurko-uwu @SparksSkywere @QuintenQVD0 @ascensionfm @AptGetGnome @kirankunigiri @cwerdna19 @Wancerz @WojenHere @DaanSelen @ThomasHineXYZ @aaddyy227 @arty01238 @Zereah Hello there! All of you should be having the same issue currently with Muse being unable to play music at this moment, currently... this seems like an issue with several things. Either ffmpeg being annoying, ytdl being annoying or some other issues that I currently can't find. However, in this case. It seems like YTDL seems to be the most damning issue at this moment. As it seems like we are not the only one with the issue, and timelines more or less match up with distubejs/ytdl-core#157 distubejs/ytdl-core#156 and distubejs/ytdl-core#153 If you are able to, please do the following. Add Caution MAKE SURE YOUR TOKEN/API KEY IS NOT VISIBLE IN THE LOG Additionally, if you want to take a gander, try upgrading/downgrading muse, ytdl-core and/or ffmpeg to see if this resolves your issue. As several issues are going on at a time, and I can't test all of them. |
I have tried to update every npm module as far as I can without code errors. It does not resolve it. |
I weirdly do not have the issue... what should be happening? |
What is your current env? Muse version, FFMPEG, distube, etc. Where are you hosting (and so on and so forth). |
Running inside Kubernetes 1.31.3
https://github.com/museofficial/muse/pkgs/container/muse I don't think anything else is installed by me other than the information for running. |
I ended up writing my own music bot because of this not working. While doing that I discovered that the latest release of @distube/ytdl-core, 4.15.8, is not working apparently universally. I downgraded to 4.15.4, and my bot worked. I kind of gave up on muse, and started to find justifications for my own (scuffed) bot, like there being muse dependencies that have not been updated for over 2 years, and have severe vulnerabilities. Like a VM2 dependency. Not really sure if there is a real concern for these dependency vulnerabilities. Coming back to muse, it works fine now with @distube/[email protected]. Basically, if I had probably just investigated a little bit I would have found that I needed to update ytdl (and downgrade? because the current muse package.json version is 4.14.4), but there were no errors or logs, just a silent failure. I had also apparently had issues with using different versions of node. Anyway, to verify, I stashed my changes and checked out muse v2.10.0 again to avoid any stupid package version changes I did. Just soapboxing sorry. From my experience, it appears to be working now. I didn't record which version of @distube/ytdl-core was being grabbed before. It was probably a ytdl-core issue, but I had convoluted the issue by changing dependency versions, and changing node versions. Edit2: Packages:
ffmpeg:
npm
node
OS: Ubuntu 22.04.5 LTS Sorry, should've included my env earlier. Hope this helps. Edit3: Just tested 4.15.8 for fun. It works.
I learned a lot about using node from all this, but I suppose it's possible I'm doing something wrong and using a version of ytdl installed somewhere else. I also confirmed that 4.15.8 does not work with the other music bot I made. The same behavior occurs; the bot joins then immediately leaves. I'll stop adding edits I think I confused things enough. Again, hope this helps. |
@cwerdna19 thanks for your (VERY) detailed explaination. I'll see if I can downgrade to the version you are using. And lock it there for the time being. I looked at the code. And yes, there is a TON of outdated code and packages in there. So I'm not sure how stable it's gonna be... |
I have read what is here, I also run a Teamspeak server with my own music bot there, recently I had to update the YouTube-DL.exe file I had there and it worked fine, so it sounds to me like a YouTube situation (adblockers maybe). I am running this bare metal on Windows Sever along with my Teamspeak bot in windows services. Following cwerden's comments I have tried running "4.15.4" of ytdl-core but sadly no audio, this debug however shows the video did somewhat parse but I did not hear anything in the VC with the bot.
I also did a debug with the current environment which is below with the latest youtube core "4.15.8" and all other modules up to date:
|
Just pulled services: Tried 3 commands: /play never gonna give you up
/play https://www.youtube.com/watch?v=dQw4w9WgXcQ
Bot joined voice channel, waited few seconds and disconnected. /play https://open.spotify.com/track/4PTG3Z6ehGkBFwjybzWkR8 Distro: Debian 12.8 Debug log in attachment |
I have a completely working installation of Muse at the moment. It plays music perfectly fine from Youtube and everything I've tried doing with it so far has worked without a hitch. But I'm going to apologize for my ignorance before I go any further. I am extremely inexperienced to this level of CLI Application complexity (and this is pretty simple stuff I'm sure). Most of what's going on in this thread is way over my head. I found this bot on a whim while looking for a replacement to JMusicBot. , and it took me hours to even get the basic installation done through Docker. I couldn't even get the install command:
to work no matter what I tried. I ended up cloning the latest Muse repository and building the image locally somehow, inputting the env variables in during runtime, and it just magically worked. I'd love to help you guys troubleshoot...whatever it is that's going on... especially since I have a fully functional Muse installation as of this very moment. But I was genuinely unable to figure out how to even create an ENV file for the Docker container in the first place, much less get it to debug mode... So that's the level of novice we're working with. I glanced at the Node is either:
or:
ffmpeg is either:
or:
ytdl-core only appears once. Not sure if ytsr is of any significance but there it is.
If I can help further let me know, but don't waste more effort on me than it's worth. Sorry folks >.<. |
@WojenHere think you could try by going to the latest branch and doing the same above, or by using a different api key. |
@Codixer just tried again latest with the same above and pr-1182. Both with different new api keys. Still the same |
@Zereah Can you upload this somewhere and send a link to download? |
@WojenHere Sure thing. Haven't used a file share site in some time, hope MEGA is ok. https://mega.nz/file/U0dmzYYA#L0VLVvoHpKmMstNNDYsZCLn6AITs2zXpAXzgJhwR-sA Hope it helps out in some way. |
@Zereah |
Has a new build been pushed to DockerHub yet at all? Currently facing the playback issues and using latest image tag. |
@WojenHere Sorry for the late reply got busy with IRL and didn't have time to sit down and figure out Docker Hub. I pushed a newly built image that is also working to my hub here. Maybe you'll have better luck with that one? Let me know if there's anything else I can try to help with. |
Huh. I wonder why that is. Somehow I've got muse working, but only when it's my built image ran with my Docker and using my API keys to join and play on my server? I'll leave it to the big brain folks to figure it out I guess. As always, let me know if I can maybe help at all, but mine just sounds like a fluke at this point... |
Trying to figure out why YT's being a bitch. But I'm getting stumped. It's like, working but in a superposition of both states of not working and working. |
This is Schrodingers Muse |
@Codixer Bit of info I stumbled into that may or may not be useful. Now having difficulty getting it to work in a friend's discord server, so I nuked the container/image and rebuilt using the github dump I still had. Tested it in my personal server first and it played music flawlessly. However upon moving the test back to the friend discord it broke again, so I scoured through the log trying to find anything notably different between the successful attempt and the failed one, and noticed this:
I also noticed the cached yt video link is broken in the failed attempt, and fully functioning in the success. I realize that might be directly related to above warning/code failure, but I wanted to bring it up just in case it was somehow relevant. Full log attached. Referenced code is at Line 574 and Line 1177, respectively. Best of luck getting to the bottom of it! |
I downgraded to 4.15.4 today as well and videos that were failing on the latest ytdl-core are working now. When I was investigating failed url's I was also experiencing 403's with certain links. Here's a link that failed on latest, Here's the same link on 4.15.4, I've redacted my IP from both URL's, but I realize that the URL that worked for me doesn't work if I redact the IP from it. I was taking a look at ytdl-core changes and it looks like they added a web client to the playerClients. Could that have broken how muse interacts with the library? distubejs/ytdl-core@4.15.4...4.15.8 I'm new to this project, so I'm still looking around the code base. Happy to help get it running, I need a new D&D music bot :P |
I forked and created a branch where I reset the player clients using the latest version of ytdl 4.15.8 and it fixed the following link which was broken on master. https://www.youtube.com/watch?v=iNPzc932cs8 I'm hoping it works on more than just my machine, lmk if it doesn't. ghcr.io/museofficial/muse:pr-1189 |
still got same error on pr-1189:
|
@WojenHere Have you cleared your local data and deleted the container? I'm updated the ytsr package now as well. Please try again using, ghcr.io/museofficial/muse:pr-1189 You will have to pull the image again to get the latest build, timestamp is around 2:24 CST. I tried your test scenarios with querying and pasting links and both worked for me. |
@SparksSkywere If you're running on Windows, can you share what nodejs version you are using. I can try to replicate on my windows machine. Have you tried running via docker or WSL? ( That's my runtime env right now ) |
@enigodupont my Node.JS version is 22.12.0, I am not going to be using docker or WSL as if we fix the issue for just docker then no one else who has a different OS can use it. |
I'm surprised the install worked with 22.12, I am getting install errors due to OPUS not having compatible node packages for 22.12.0. The doc should say Node 18 is required as OPUS has versions available for version 18 and 19. https://github.com/discordjs/opus/releases I was able to build with v18.20.6 and run the bot on windows. I've been getting issues with the formats not being pulled by ytdl-core so I'll look into that. It's not working for me anymore since the API isn't returning formats :( |
I found that ytdl-core seems to have busted how formats are parsed when using the old default player clients. I also found that when we do obtain formats, the OPUS url's tend to get 403's. EDIT: Not all other media types work fine, I'm getting mixed results when I try to click them. Sometimes they work, sometimes they don't. |
Wanted to jump in and say that I'm experiencing the issues mentioned in #1173 #1166 using the pr-1189 image. Here's the logfile: https://0x0.st/8XoH.txt |
@enigodupont The funny thing about why my version is this is because on the same day as my final server migration was going on moving to the new host I was getting all subsystems back in and happens the Discord bot stopped working, I spent a couple days just checking if it was my server! |
Wouldn't be better to migrate from |
If you know someone who has experience with yt-dlp code, please help xD |
What an utterly bizarre journey this has been, especially as such a novice. My previously working bot using a much older branch has now also stopped working, experiencing the same issues as most others, #1173 #1166, despite ostensibly changing nothing with the setup. It appears to be malformed/broken URLs from either ytdl or ffmpeg, as they do not lead anywhere when clicked. The last time a bot worked, I could click on the cached URL and it was a valid embed playing. But I don't understand why a previously working image would suddenly stop working with no alterations on my end. @enigodupont 's pr-1189 image is also non-functioning for me, but is returning a 🚫 ope: Can't find suitable format. without even spawning ffmpeg first. Not sure which one is even closer to working at this point. Just sharing my experience in hopes it leads someone to something. |
If I had to guess, I'd assume api calls are breaking because youtube is changing their API's right now and ytdl-core is working on accommodating the changes. I'm following this for updates, so once their changes are merged I can try to incorporate them. Looks like playerclients and how formats are handled are undergoing significant changes. |
I pulled in the changes upstream in this PR, #1197 As of writing it's working for me in my servers. Please give it a try when y'all can. ghcr.io/museofficial/muse:pr-1197 |
@enigodupont still not working for me on docker compose. Checked with new discord bot profile on few discord servers and on a new server |
I just tried to use commands without autocomplete (/clear and /config get). Still shows "App is not responding" on discord and Unknown interaction in bot logs |
Hmm, you're running this on a homelab somewhere right? Does this happen on all machines you run this on? |
Maybe I used the wrong words when calling my machine ‘homelab’. By 'homelab' i mean my little dell optiplex connected to my router where I run Debian with docker, I'm not using VMs |
@enigodupont got it worked sometimes. Steps to reproduce:
Bot works very slow, it takes about a minute to "think", join voice channel and start playing. Works with typing song name and pasting youtube link. Spotify links don't work. Autocomplete does not work either, after 3 seconds shows "Loading options failed" Logs here: https://pastebin.com/zSDLCYRy |
Do you have other machines available for testing purposes? What happens if you install node and yarn, then run locally using yarn install + yarn start? |
Interesting, I'm wonder if docker is putting limits on your container that's causing it to run too slowly for discord api's to respond quickly enough? I was doing some testing on my homelab and I found that performance dipped when I gave it .25 CPU and 128Mi of RAM. I found a sweet-spot with .5 CPU and 400Mi of ram. |
Can't help right now, The earliest I will be home is Tuesday. I can check it but not in the same network and device. |
set to 2 cpu and 1024m ram, still the same result |
@enigodupont Sadly the issue persists, I did a fresh install again, my only thought is ffmpeg being bad or something along those lines. |
Here's what worked for me on WIN 11, using powershell to run "yarn install and yarn start".
|
I am using npm install, let me give yarn a go now |
@enigodupont Same result for both NPM and YARN, however I have a special cmd file I use to start the npm as it will cause the error "muse Error: Cannot find module 'node-opus'" otherwise, I wonder if my ffmpeg is broken! What version of ffmpeg are you using? Mine is "ffmpeg version 2024-12-26-git-fe04b93afa-essentials_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers |
I have the following
I recommend using the following node,
EDIT: I do think to get it working you need to download node-opus which requires node 18 EDIT: This information is from my local install on windows, not docker. |
@enigodupont Muse is now working for me in all servers using pr-1197 and no other changes. I'm still using an extremely old version of ffmpeg but I'm too dumb to figure out how to run the pulled image with newer version I have that you suggested, or where it keeps getting that version from, but it's working anyway so I'm afraid to touch anything. |
Happy to hear its working! I added some context in my previous comment, but the ffmpeg I'm running is on my windows machine. The one you are running is in docker and I believe I run the same thing when I run docker. Should work with both versions so I wouldn't worry about it for now. |
Thanks for the clarification, not sure why Docker is using a version that's 4 or 5 releases behind, but that's a problem for another time. |
I GOT IT TO PLAY! I went and gone through my home PC for any old codex's I had, now I had a copy of ffmpeg from 2021 for some old copies of my teamspeak and audacity files with these .exe files, after that I made a new folder and inputted all the usual install scripts but did a full upgrade with forcing any and all to upgrade, after doing so it worked!
I then re-upgraded the ffmpeg back to the latest copy I had and it still worked! My only guess was something to do with the npm modules? But either way I have it all working now :D |
Update: I have done further experiments, the 2021 version of ffmpeg I have works better than the latest |
Closes #1173
Closes #1171
Closes #1166
Closes #1165
Closes #1163