Add 2-bay HDD enclosure support #8
Conversation
… the second drive is for vWii. WiiU will recognize only the 1st drive but ignore the 2nd, which works in our favor - WiiU won't even nag you to format the 2nd drive to its proprietary format. usbtoggle is not required. Both drives are adequately powered by an external power brick because all 2-bay enclosures are externally powered. No more Y-cables. (WiiU is very strict about the 500mA USB current limit.) If you have a RAID enclosure which can be set to RAID 0/1/JBOD/2LUN, set it to 2LUN. On a computer 2LUN simply shows as 2 separate hard drives. This cIOS works only if the base IOS is either 56 or 57. base58 is not supported (because it does not patch the EHCI module). Your homebrew shall remain on an SD card. The WiiU side demands that payload.elf be on an SD card and we can put all vWii homebrews on the same SD card, too. Most homebrews run with IOS 58 which does NOT see the second HDD. In USB Loader GX, set USB Port to 1 to choose the second hard drive. You may have to restart for the change to take effect. Until now, the USB Port setting only works with an 'alt' cIOS such as d2x-v10-beta53-alt. The usual advice is to use non-alt cIOS and plug the hard drive to USB port 0. That advice still holds, the only change being that the same USB port 0 is now home to 2 HDDs, with the first hard drive appearing as 'USB Port 0' and the second hard drive appearing as 'USB Port 1'. I have chosen to repurpose USB port selection as LUN selection so that no changes need to be made to USB Loader GX. The other solution considered was to scan LUNs in reverse order but that was rejected because I want no behavioural change compared to standard cIOS (d2x-v10-beta52) unless ordered by the user. Also considered was automatically detecting if the drive belongs to WiiU and choose the next drive if that is the case. That requires verifying some signature bytes on the disk and I do not wish to take risk on the initialization process.
…oups in d2x-cios-installer. In my opinion this is the less risky solution in the wrong hands. Leseratte added console= and region= tags to ciosmaps.xml. A new d2x-cios-installer is needed with enhancements to support those new tags and filter the user interface so that only the options pertinent to the user's hardware are shown. However, nothing stops users from using this cIOS with existing the d2x-cios-installer with no such support, and then proceed to install the wrong cIOS on the wrong console (e.g. Wii cIOS on vWii). The safest thing to do IMO is to group them into ciosgroups. Time comes when somebody makes a new d2x-cios-installer it can still support Leseratte's new tags and filter the options. The only cost to this arrangement is file duplication and extra disk/SD card space.
|
Wow. I'll probably need a while to "get back into" the code again to properly understand all these changes, but it's great seeing people taking up cIOS development again, given that I don't have as much time available for cIOS development as I'd like. Thanks for the PR! So, just so I understand this correctly, this basically makes it so that if there's one USB device that "exports" multiple drives, this makes it so that it always opens the last one? Given that the development of the HBC and all existing USB Loaders has basically stopped, I don't think there will ever be a way to signal from the USB Loader to the cIOS which device to mount, but have you thought about maybe doing something like checking all attached devices for a given file / identifier to determine which one to mount? Also, does this PR "just" support single USB devices with two logical drives, or would it also support attaching a USB hub to the Wii and then attaching multiple regular USB devices to that hub? I'm guessing it won't, but do you have an idea how difficult it would be to add that? |
…ess user authorizes doing so by explicitly enabling USB Port Both or USB Port=1.
|
I've considered several options such as the ones you surmised. Any cIOS mod shall be drop-in replacements but not require any changes to the homebrew apps on top. (So any solution requiring a new signaling mechanism is rejected.) Let you said, HBC development is not very active nowadays. Second, while 2-bay primarily benefits Wii U users, Wii users must be able to just take this cIOS and use it, and to them, there shall be no changes in existing behaviour. So the reverse scan order I originally proposed was rejected, too. If IOS 58 scans forward but had my cIOS scanned drives backward it would have been very difficult to comprehend when both IOS's are mixed and used on the same console. As for hub, it sounded like something that v9/v10-beta53-alt did or would be able to do, but I didn't go in that direction because alt did not have a good reputation. I was not around the scene back then but I guess what happened was that a lot of efforts were spent on v9 building plug-and-play, multi-ports, etc. which negatively impacted compatibility and v10 (non-alt) went back to the v8 code-base. People saw no point continuing down the v9 path given that Nintendo had IOS 58. Given the backstory, following the v9 lineage was not a risk I would take, so no hub support. However, on a regular Wii I tested dual HDD on USB port 0 and FAKEMOTE on USB port 1, and that worked!!! So the solution I chose was to repurpose the v9/v10-alt signaling mechanism. USB Loader GX and WiiXplorer use that to switch between 2 hard drives on 2 separate USB ports, but I take that to mean 2 hard drives on the same USB Port 0. The underlying cIOS implementation is very different from v9 (and a lot simpler) but the API to activate the second drive is the same. In USB Loader GX just change the "USB Port" option to "1" or "Both" (from 0). In WiiXplorer there is an option to use both USB ports. Some UI changes to USB Loader GX would help, but not necessary. For instance the option could be called "USB Drive" instead of "USB Port". The options remain "0", "1" or "Both". If 2 separate USB HDDs are used with v10-beta53-alt, choose the appropriate "USB Drive" and for a single enclosure with 2 HDD bays, also choose the appropriate "USB Drive". Same wording. That's only a UI clarification with no need of any changes to the actual code. I did not know how the Reload IOS Block started working on the Wii Mini. I did not do anything to fix it, but it fixed itself. I speculate that the process of disk initialization/dismounting had some corruption that required a reload to correct, but by cleaning up initialization/unmounting code, that restart is now no longer required. In my opinion this alone deserves a "release". I have solicited the community's help with testing. I have not garnered any feedback from anyone who has actually tried it out yet but I am very hopeful that given some time, we will get tested. In the meantime my son will use it as his daily cIOS to ensure the best stability. I probably won't be able to do anything for decade-old issues but at least I must make sure there shall be no regression. Before I retired, I did code reviews at work and it was face-to-face or by video conference. That way I would have to explain the code and questions/feedback would be immediate. The direct interaction helped very much to refresh memory of code that you worked on some time ago. If that sounds like it would help, let me know. |
|
Thanks for your detailed response. As for cIOS mods being a drop-in replacement, I definitely agree with that - but it also means that every stupid design decision from 10 years ago will need to be carried over, even if there might be better (or more compatible) solutions available now. HBC and homebrew development sadly stopped a long time ago. If I have some time to dedicate to cIOS development in the future, maybe I'll try adding something like a config file. Make a small homebrew app that writes a cIOS config file to the NAND, and have the cIOS read that on each initialization to trigger different behaviour. That way it could be implemented to behave exactly like the old cIOS when no config file is found, but if the file is present, people could do additional stuff like switch drive order, choose which drive to use, or any other stuff that might still be added in the future. And more importantly, these config changes done by the user would then also apply to old USB Loaders. I mean, you had to change the version number from 1 to 9 to get loaders to support dual HDD, and maybe that's also what fixed IOS Reload - stupid loaders gatekeeping functionality behind a cIOS version number check. That's also why I named the cIOS "d2xl" instead of a different name, as there's some loaders that check for d2x compatibility by checking if the version string starts with "d2x". As for code reviews refreshing code I worked on a long time ago, that would be a great idea if there was any code in the cIOS that I worked on :P - basically, I never had anything to do with cIOS development. When I started working on any kind of serious Wii modding, cIOS development had already ended, so what I did to the cIOS so far was getting it to compile again, adding a hack fix for Wii Mini support, and just playing around with the code in general to try and figure out how it works. And then I got sidetracked again with all the other projects (Wiimmfi, LE-CODE, other non-Wii-related projects ...) that I'm working on. Though, interestingly there's been a new commit (and release) on the old repo by davebaol regarding UStealth support (#9), so I'll have to take a look at that, too. Didn't think there would ever be any new updates to that repo again. |
|
I looked at the FAT interface briefly. There can be at most 8 USB partitions, named usb1: to usb8:. Nothing written on stone says they must come from the same disk. We do a scan to find partitions on as many disks as we can, over any number of drive bays and any number of USB ports... and to the user it is just a path. That would have been ideal, and transparent to existing applications. But the problem is we have to support the Read Sector and Write Sector interfaces in the EHCI module, and those have no concept whatsoever of devices, disks, ports or LUNs. It has always been assumed to be one disk. The Set Port extension is not very pretty, but our hands are tied. We are not able to do fundamentally better because of Read/Write Sector. They take only a 32-bit LBA sector number addressing up to a 2TB disk and that's the hard limit. If an extension is created to address larger disks and multiple disks, we will have to create a new suite of incompatible apps. If everything goes through FAT then we would have had a nice user experience and support this transparently without the user's involvement. I wonder if you have any plans to PR Leseratte10/d2x-cios to davebaol/d2x-cios. Just asking... I've marked the binaries on 2022-02-28 as "v2". Let me try and see if I get a black screen from that. |
|
Exactly, because there's just the one "set port" API, things are going to get ugly. But if the cIOS were to support USB hubs and everything and be able to initialize however many drives a user has connected to their Wii, then a cIOS config file (that can be written by a homebrew app) could then be used to be like "okay, the drive with vendor ID X and product ID Y and serial number Z is considered 'drive 0' no matter where it's connected to, and the drive with vendor ID ABC is considered 'drive 1'", and then people could do stuff like attach a USB hub to their Wii Mini and add up to two useable USB drives plus however many Wii accesssories they want to that hub. As for PRing my changes in this repo back to davebaol's repository, I did not plan to do that as it seems like development over there is completely dead. The last actual code changes were in 2012 (to add WiiU support), and after that nothing has happened over there. That's almost 10 years without any updates. In 2019 (when I softmodded my Wii Mini with Bluebomb) I opened a bug report over there asking how to compile the cIOS (as nobody I asked knew how to properly compile them). I did get a response from davebaol after pinging him, but even with that the project seemed pretty dead. When I started working on my fork back in 2019 I was hoping to have a bit more time available to dedicate to the cIOS, but that didn't happen. But still, considering that development in the original repository has ceased a long time ago, I don't see a good reason to merge my fork back into the original repository, having to rely on davebaol to merge future changes or to make new releases. |
|
Just tried "v2" (2022-02-28). Reload IOS Block still functioned without any black screen! WiiXplorer says d2x cios v9+ required. This is the first time I work on open source. The dynamics are new to me. Forking is easy but there's got to be a central repository to integrate work from many different people. Whoever is the most active would serve as that central repository, and such is the democratic nature of open source. If there are no plans to merge back to the origin d2x I totally support your decision. I think d2xl can carry the banner going forward. The foundation is very solid with the whole thing on docker and CI. To the user d2xl is the go-to source for Wii Mini, and I am adding work that is of interest to some Wii U owners. It does not need to be d2x's replacement or successor, it just needs to offer a compelling alternative to certain classes of users. I have no ambition to expand my cIOS work any work further, but I believe what we have in hand is useful and deserve a release if sufficiently tested. Progress will be slow and work will be sparse, but I think that will be fine. |
…ver 20 retries, then give up instead of hanging.
…he hard drive awake but it does not have to move the head across the platter that much. A close-by sector will do. 2, change retry to increase timeout exponentially, from 2, 4, 8 to 64 seconds, then give up. Minimum is uniformly 2 seconds based on a WD Scorpio Blue 1TB that I pulled from a WD NAS/Router.
…g thread. The drive's cache cannot accuse us of rereading the same sector over and over again and spin down the drive.
…S Router which appears to have customized firmware. It spins down readily making it an ideal test subject for power management issues. The watchdog thread will keep the current drive active while the other drive is idle and should be allowed to standby. I listened to the drive spin down with a stethoscope and then do something to make it spin up again to see if anything breaks. Tried a few things in USB Loader GX like dumping NAND, backing up a retail disc, and in WiiXplorer copying files between the 2 drives while one of them is sleeping.
…ci_reset_port2() instead.
…rage_reset() calls __cycle() to do stuff that has the potential to timeout. Infinite reentrancy. With that out of the way, I am reverting most of the changes making most of the code exactly the same as before (d2x-v10), keeping the fixes that are demostrably helpful. Namely, some timeouts are increased, and timeout keeps on increasing as retries are attempted, so that we won't run into a vicious cycle where some operation is ramping up but we keep killing it and restarting it because the timeout we give is too stingy. Secondly, the error handling for an interface failure and a single drive timeout are separate cases. The former continues to use the nuclear strategy of unplugging and hard-resetting the USB interface, but that shall not be applied to the latter. We must not kill the innocent along with the guilty. A soft reset is the appropriate way to deal with a single drive timeout.
… any more but simply fail. __cycle may cause a soft reset but a soft reset don't need __cycle. Still had to remove the part where if a reset fails it dismounts and tells __cycle to fail too. Don't know why but still need to investigate. But for now, it works and it doesn't hang.
https://gbatemp.net/threads/2-drive-enclosure.608112/