Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MEGA65: adopt more decent hyppo #140

Closed
lgblgblgb opened this issue Sep 3, 2020 · 16 comments · Fixed by #336
Closed

MEGA65: adopt more decent hyppo #140

lgblgblgb opened this issue Sep 3, 2020 · 16 comments · Fixed by #336

Comments

@lgblgblgb
Copy link
Owner

lgblgblgb commented Sep 3, 2020

Project status:

Highest priority for now

So there should be some SD-controller emulation problem, that hickup thinks now, it's not an SDHC card. Though Xemu only emulates SDHC card. As hyppo seems to detect SDHC by trying non-sector aligned access (as with non-SDHC cards the addressing is byte based, but must be sector aligned, thus a method to detect it), but it should work anyway. So I assume, there is some generic problem with SD-controller emulation that a regular SD-sector read command fail, when it shouldn't.

Jimbo on MEGA65/Xemu discord suggested a two line hack for keeping status read busy when external bus is selected for SD operation. This seems did the trick, however the whole SD-controller emulation is very messy ...

TODO list:

  • Avoid the QSPI problem somehow ...
  • Fix the hypervisor-execution check error message, which always kicks-in now
  • Utility menu cannot be accessed any-more, (emulated) MEGA65 says, you must power-cycle ...
  • "Boarding utility" kicks in and freezes with green screen after saving, though after a restart, system seems to work fine!
  • Probably update Xemu's internal config save to avoid this and bypass onboarding utilities as well? Not sure, if it's a good idea or not ...
  • Remove some old crap (like C000-utils?)
  • Utility menu items (configure, fdisk, keyboad test) causes a hang (at least a white screen with black border). Worked before with older hyppo
  • Cleanup: decide if chrWOM is 4K only on MEGA65 or not (C65 in theory supported dual charsets). Also, what part of ROM does hickup uploads to WOM? Fix all cases of WOM filling access with the correct policy.
  • Remove -fonrefresh option and make it compulsory part of the code if custom ROM is used.
  • Is CHARROM file needed at all on the SD-card?
  • Alter code to allow to specify custom system files, simplify/cleanup code.
  • Look after the problem that the bottom of the screen is missing (though it seems only in NTSC mode ...) UPDATE: this may be a real issue, however hard to reproduce, and according to others, sometimes similar symptoms (not sure, if they're the same though) can be seen on the hardware as well. So let's not deal with this, at least for now.
  • Remove, or alter hickup debug code, which would still expect Ophis-specific files, so quite useless now. Probably it won't hurt if remains, but would be not very hard to use acme's symbol file as well (since hyppo went away from using Ophis to Acme as the assembler!)
  • Probably rename everything "kick" to "hick" in Xemu (also options)
  • Update sd-card updater code to populate with new system files
  • Bump the meminit version number, if everything is OK, so Xemu will detect and suggest users to update their sd-card image if it's an older installation
  • Revise, how "chrwom" data is extracted from VHDL, a binary file would be much nicer
  • Revise all the tricks/hacks of the "first time hypervisor leave" hook
  • Revise/fix/cleanup how ROM is loaded, ie user controllable custom ROM, and do not collide with the pre-init openrom
  • Maybe allow Xemu to boot into open-rom built-in just for fun (not so high priority)
  • Maybe add an UI option to temporary reset with an external ROM without the need to update SD-card
  • Add hypervisor version report
  • Add HDOS hooks so Xemu can check and/or override HDOS traps
  • CPU speed control seems to be bad now :( Some description and ideas: MEGA65: adopt more decent hyppo #140 (comment)
  • "Downgrade" too many, too detailed console log messaged (DEBUGPRINT instead of DEBUG) and set specific stuffs (like DEBUGHDOS back to alias for DEBUG rather than DEBUGPRINT). Especially in sdcard.c and hdos.c, but maybe others, in general! vic4c itself does too much DEBUGPRINT as well, however it's not the scope of this issue too much
  • HDOS virtualization should be finished at a level at least, that BASIC65 new things works, even if not all the HDOS functions are implemented. However it shouldn't be made default as other software (even like GEOS65 or possible others) may do other things not virtualized at all, thus causing "interesting" problems then with HDOS virtualization turned on. ** Moved to MEGA65: hyppo HDOS virtualization #331 **
  • Restore the logic of "default D81 is outside of the SD-card" functionality. Maybe check if MEGA65.D81 is on the SD-card's root dir, record its starting SD sector number and use external source if internal mount finds matching mount sector with that. So it would be "kinda transparent".
  • Remove the ugly "Xemu information query" method done in the I/O decoder ... I guess nobody used it too much (other than my info utility ...) and it's now preferred to use the Xemu specific trap to get emulation related information (also much cleaner, and much easier to implement and also to use!)
  • Consider supporting D65 "superfloppy" images: according to Paul: "64 sectors x 85 tracks in size, raw"
  • Consider supporting D64 with on-line translation to D81, maybe in R/O mode only though! External mount only, since there is no real hardware equivalent of this on MEGA65 to support via registers! (possible IEC drive emulation in future in Xemu is a totally different topic!)
  • Support for D64 and D71 images, mounted as D81 (as per geometry) surely with checking the image size violations. The actual geometry conversion is need to be done in software (ie, by ROM, for example)
  • Go for the MEGA65-release stage of hyppo and system files! Now I use hwsc which is maybe buggy and would be better to match with real MEGA65 expected to be shipped with, rather than some other branch! So the master branch.
  • Matrix-mode control in Xemu (key trigger) should not use a hypervisor trap as it would make it impossible to toggle matrix mode when in hypervisor mode. Surely, in Xemu UI it's still possible, but let's make the life easier and more consistent.
  • possible some anomalies on mounting D81s, sometimes it does not work to mount new image or even revering back to SD-card default image (yes, even when external D81 default image is active). Fix this and test as much cases as possible to see there is really no bug left here
  • normalize UI menu a bit for D81 mounts, too many options, maybe divide the new disk menu (also contains sd-card submenu) for drive8 and 9 and have the options there, specifically at least "mount + use", "create + use", "detach", also maybe an option for mounting default for drive8 (but then it should matter if D81 from SD-card is enabled or not!)
  • possibly implement some missing SD-card commands reported as missing, also read back of some sd-controller registers (for example reported by GEOS65)
  • in case of external mount (including internal d81 replaced with external hack) let's implement register modification to signal it's external for real. I am not sure if it is not a problem and better to left as it is? But not consistent this way ...
@lgblgblgb
Copy link
Owner Author

MEGA65/open-roms#54

Reason why I may want to delay adopting a new Hyppo at least this 'early'.

@lgblgblgb
Copy link
Owner Author

lgblgblgb commented Dec 6, 2021

A more fresh experiment, still the BRK problem:

CPU: at $AADB opcode = $A9 LDA #$nn A=FF X=00 Y=00 Z=3F SP=FD
CPU: at $AADD opcode = $8D STA $nnnn A=AA X=00 Y=00 Z=3F SP=FD
CPU: at $AAE0 opcode = $A9 LDA #$nn A=AA X=00 Y=00 Z=3F SP=FD
CPU: at $AAE2 opcode = $8D STA $nnnn A=0F X=00 Y=00 Z=3F SP=FD
CPU: at $AAE5 opcode = $A9 LDA #$nn A=0F X=00 Y=00 Z=3F SP=FD
CPU: at $AAE7 opcode = $8D STA $nnnn A=FF X=00 Y=00 Z=3F SP=FD
CPU: at $AAEA opcode = $A9 LDA #$nn A=FF X=00 Y=00 Z=3F SP=FD
CPU: at $AAEC opcode = $8D STA $nnnn A=F0 X=00 Y=00 Z=3F SP=FD
CPU: at $AAEF opcode = $60 RTS  A=F0 X=00 Y=00 Z=3F SP=FD
CPU: SP before RTS is (SPHI=$0100) SP=$FD
CPU: at $AFC0 opcode = $4C JMP $nnnn A=F0 X=00 Y=00 Z=3F SP=FF
CPU: at $080D opcode = $00 BRK  A=F0 X=00 Y=00 Z=3F SP=FF
CPU: WARN: BRK is about executing at PC=$080D

After this on the screen (no cursor, or ability to type):

xemu

Also before this:

execution outside of the hypervisor memory, PC = $5796

and (many similar):

WARNING: Unhandled memory write operation for linear address $FF10005 data = $D7 (PC=$761E)
CPU: at $761E opcode = $8E STX $nnnn A=D7 X=81 Y=FF Z=3F SP=F5
WARNING: Unhandled memory write operation for linear address $FF10006 data = $81 (PC=$7621)
CPU: at $7621 opcode = $8C STY $nnnn A=D7 X=81 Y=FF Z=3F SP=F5
WARNING: Unhandled memory write operation for linear address $FF10007 data = $FF (PC=$7624)
CPU: at $7624 opcode = $9C STZ $nnnn A=D7 X=81 Y=FF Z=3F SP=F5
WARNING: Unhandled memory write operation for linear address $FF10008 data = $3F (PC=$7627)
CPU: at $7627 opcode = $A2 LDX #$nn A=D7 X=81 Y=FF Z=3F SP=F5
CPU: at $7629 opcode = $BD LDA $nnnn,X A=D7 X=04 Y=FF Z=3F SP=F5
CPU: at $762C opcode = $9D STA $nnnn,X A=53 X=04 Y=FF Z=3F SP=F5
WARNING: Unhandled memory write operation for linear address $FF10004 data = $53 (PC=$762F)
CPU: at $762F opcode = $CA DEX  A=53 X=04 Y=FF Z=3F SP=F5
CPU: at $7630 opcode = $10 BPL $rr A=53 X=03 Y=FF Z=3F SP=F5
CPU: at $7629 opcode = $BD LDA $nnnn,X A=53 X=03 Y=FF Z=3F SP=F5
CPU: at $762C opcode = $9D STA $nnnn,X A=4F X=03 Y=FF Z=3F SP=F5
WARNING: Unhandled memory write operation for linear address $FF10003 data = $4F (PC=$762F)

@lgblgblgb
Copy link
Owner Author

lgblgblgb commented Dec 6, 2021

Ok, what I've learnt today from Paul:

On Discord: https://discord.com/channels/719326990221574164/781481205639020554/917518905285685368

there are bunch of memory regions must be pre-filled with the right content (filled with the bitstream on MEGA65), only open-rom is not enough. The reason of the BRK, that hyppo tries to execute the flash menu, but it was not loaded, so only zero bytes there, thus the BRK problem occurs. Also, the check of hypervisor running outside of hypervisor RAM is largely outdated, as for now hickup does this by will, though maybe only the "boot" process, and after the first return from the first hypervisor call. it's still valid to have the check.

I may open a new branch to play with newer hyppo, as it seems it requires "some" re-work here and there, it won't be as easy to replace the hyppo image only. The new branch for the experiment: hyppo

Important "map file" for hyppo for debugging: src/hyppo/HICKUP.rep

Recommended mega65-core branch to get files from: master or hwsc (more bleeding edge)

Needed pre-initialized memory areas (according to Makefile):

$(TOOLDIR)/mempacker/mempacker_new -n shadowram -s 393215 -f $(VHDLSRCDIR)/shadowram-a100t.vhdl \
    $(SDCARD_DIR)/BANNER.M65@57D00 \
    $(SDCARD_DIR)/FREEZER.M65@12000 \
    $(SRCDIR)/open-roms/bin/mega65.rom@20000 \
    $(SDCARD_DIR)/ONBOARD.M65@40000 \
    $(UTILDIR)/megaflash-a100t.prg@50000

That it seems:

File address Comment
FREEZER.M65 $12000
open-rom $20000 used for boot process, will be overwritten by Hyppo from SD-card file MEGA65.ROM
ONBOARD.M65 $40000 boarding utility, ie setting initial parameters by user once, and store on SD card
megaflash-a100t.prg $50000 Flash-menu: not used in xemu (pointless), but a fake, xemu specific stub only
BANNER.M65 $57D00 the MEGA65 logo

NOTE: other files should be put on the emulated sd-card as well, as those may be loaded (once working freezer is a reality in Xemu) like sprite editor and things like that. TODO: get full list of needed files, let it be either memory-initialization purposes, or other.

@lgblgblgb
Copy link
Owner Author

lgblgblgb commented Dec 8, 2021

Initial work committed to branch hyppo using brand new hickup (and other "system files") from the bleeding edge hwsc branch of mega65-core.

First, there is QSPI-flash issues, which is expected (this must be handled somehow, as the error is not fatal, but it's really annoying that emulator users always must to press the famous ANY key to continue, though doing so really allow to continue without any noticeable ill-effect):

xemu1

Second, there is an SD-card detection problem though:

xemu2

STATUS

This part is moved TO THE TOP of this page, since it was already confusing enough ...

@lgblgblgb
Copy link
Owner Author

lgblgblgb commented Dec 9, 2021

Idea about QSPI: it seems hickup just does a JMP $80D or so, to jump to the flash utility. That one exit with JMP $CF7F otherwise (like when printing that error). So, in theory, by using some own "stub-QSPI-flash-util" which is only that JMP opcode to $CF7F would may do the trick, so I don't need custom hickup for Xemu but avoid to dealing with the flash utility as much as possible.

DONE, a minimal built-in program to Xemu is added to function as a "fake" version of the flash utility.

@lgblgblgb
Copy link
Owner Author

lgblgblgb commented Dec 9, 2021

ONBOARD.M65 does an FPGA reconfiguration trigger by writing $42 to $D6CF. I've just implemented this by a reset. Now it does not stuck after save settings. However I'm still unsure, if it's a good idea to leave this tool in an emulator, or not ... I may replace it with a "stub" like with the flashing utility. However it would mean, Xemu's solution to write some config out as its own must be updated, since there is such a code, but I guess it's not valid anymore, that's the reason ONBOARD kicks in.

@lgblgblgb
Copy link
Owner Author

lgblgblgb commented Dec 9, 2021

Added memory pre-init for reset routine to allow to access utility menu from emulation menu. Now the menu is displayed, however configure utility cannot be entered, only a white screen is shown. This worked before with older hyppo. The same for format/fdisk and keyboard test utilities :( I also see some BRK stuff here, after enabling in-deep debugging in Xemu to see what's happening.

lgblgblgb added a commit that referenced this issue Dec 13, 2021
lgblgblgb added a commit that referenced this issue Dec 13, 2021
lgblgblgb added a commit that referenced this issue Jan 8, 2022
lgblgblgb added a commit that referenced this issue Jan 12, 2022
lgblgblgb added a commit that referenced this issue Jan 12, 2022
The problem: I've triggered matrix mode with keypress by emitting a
hypervisor trap for that functionality. Which works fine, unless you're
already in hypervisor mode (boot process, trying to debug situation when
stuck there!). In hypervisor mode, surely it's not possible to cause a
trap. Thus my idea, is to control matrix mode directly bypassing the
trap part. Seems to work fine so far ...
lgblgblgb added a commit that referenced this issue Jan 21, 2022
As part of #140 I encountered the problem, that xemu_open_file()'s
method to do a "second mode set" open (if first failed) has a serious
design flaw: it sets the pointed value to zero if the first (default)
mode was used, however RD_ONLY is also gives zero ... Thus I introduced
the XEMU_OPEN_FILE_FIRST_MODE_USED constant (set to -1, thus it's
very unlikely it's a valid "mode" value! - ie all bits are one) instead
of zeroing. Some places of Xemu had to be modified to use the new scheme
though.
lgblgblgb added a commit that referenced this issue Feb 21, 2022
Note, that D64/D71 is mounted as D81, so the software (like ROM) need to
translate disk geometry. The problem with this commit:

* D65 support has its own geometry unlike D64/D71, but I am not sure
  what it should be!
* Currently only external mount works for any non-D81 format.

The previous "fake D64" method has been deactivated.
lgblgblgb added a commit that referenced this issue Mar 13, 2022
lgblgblgb added a commit that referenced this issue Mar 13, 2022
lgblgblgb added a commit that referenced this issue Mar 16, 2022
Also introducing (not a complete list):

* (acme) REP file based hypervisor debugging
* HDOS call tracking
* HDOS virtualization alpha stage
* System initialization changes
* Better initial D81 mount tracking
* More sane CLI/options (eg single rom option
  to load ROM, no need extra -forcerom and other
  bizarre things)
lgblgblgb added a commit that referenced this issue Mar 16, 2022
* Reogranization some of the UI menu
* Provide new functionalities
lgblgblgb added a commit that referenced this issue Mar 16, 2022
The heart of the `hyppo` branch, that is using a new and decent enough
hyppo. Also it required many changed here and there to support the new
scheme, like using an initial ROM (open-ROMs) pre-filled the memory
along with other utilities.
@lgblgblgb
Copy link
Owner Author

Branch hyppo is now merged into next (the previous next into master). Though it's not perfect, it's finally mature enough to be at least "usage material" for wider audience keeping mind, that it can still buggy.

@lgblgblgb
Copy link
Owner Author

I close this issue now, since the main goal is achieved to have a usable emulation running a decent enough hyppo, and already has been a very bumpy and long road. Individual remaining and known bugs (and possible unknowns ...) should be addressed as-is after this.

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant