Developing for MSX-DOS2 and >64kb
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Developing for MSX-DOS2 and >64kb
I have read through the example files for the SMS/zx/CPC/MSX that show the function banking features... And I think I understand how they can be made use of for a ROM target, however the game I am developing is intended for a disk target; I get that I will need to structure my application so that I have, at minimum 1 bank permanently swapped in containing essential, non swappable global data/code, with the other banks swapped in from their pragma defined bank numbers (with care in structuring the application so that I don't need mor banks swapping in than I have free at any given time - obvious candidates for being excluded from the unswappable banks would include, I assume, graphics assets and/or audio)
Are there any hooks in z88dk at present to support additional 16kb code, data or mixed code/data banks for an MSX-DOS2 target?
I haven't read nearly as much MSX technical docs as I should have, but there is reference to memory mapper functions in DOS2, but from what I can make out, this is intended for runtime allocation of additional ram. Is there something as simple as the banking example code for non-ROM targets?
Are there any hooks in z88dk at present to support additional 16kb code, data or mixed code/data banks for an MSX-DOS2 target?
I haven't read nearly as much MSX technical docs as I should have, but there is reference to memory mapper functions in DOS2, but from what I can make out, this is intended for runtime allocation of additional ram. Is there something as simple as the banking example code for non-ROM targets?
Re: Developing for MSX-DOS2 and >64kb
I too have never read that section of the MSXDOS documentation either, I've had a quick scan through and can see that it's not as trivial to do banking in a DOS environment as in a ROM environment but still very possible.
As you've spotted, banking is achieved by a few things:
a) A set of defined sections: https://github.com/z88dk/z88dk/blob/mas ... egarom.asm
b) The banked_call routine: https://github.com/z88dk/z88dk/blob/mas ... m.asm#L141
c) A loader (for non ROM targets): https://github.com/z88dk/z88dk/blob/mas ... 0.asm#L148 (that's for +zx)
The bank numbers are set at link/compile time, however DOS will give back a segment number that corresponds to free memory so you'd need to maintain a mapping between the binary bank number and the DOS segment number which is used by the banked_call code.
It all sounds very achievable, it might take me a few weeks to get round to doing it myself, so feel free to give it a go yourself.
As you've spotted, banking is achieved by a few things:
a) A set of defined sections: https://github.com/z88dk/z88dk/blob/mas ... egarom.asm
b) The banked_call routine: https://github.com/z88dk/z88dk/blob/mas ... m.asm#L141
c) A loader (for non ROM targets): https://github.com/z88dk/z88dk/blob/mas ... 0.asm#L148 (that's for +zx)
The bank numbers are set at link/compile time, however DOS will give back a segment number that corresponds to free memory so you'd need to maintain a mapping between the binary bank number and the DOS segment number which is used by the banked_call code.
It all sounds very achievable, it might take me a few weeks to get round to doing it myself, so feel free to give it a go yourself.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
Yes, it appears the DOS2 mapper functions were introduced to try to bring some level of consistency to the various mappers that were available on the MSX platform, i.e. since the RAM/ROM mapper implementations seem to have created quite a mess on the platform - DOS seems to do the interrogation of the available mappers on boot, and present them in a consistent way.
There are a couple of DOS2 function calls that make accessing the extra blocks easy, plus a couple of calls that read and write bytes directly.to those banks (withhout being swapped in, from my understanding). It sounds ideal for loading additional data into those areas, a bit like the old extended memory calls on MS-DOS, but perhaps not quite as easy to use for additional code areas as th ROM banking methods.
There are a couple of DOS2 function calls that make accessing the extra blocks easy, plus a couple of calls that read and write bytes directly.to those banks (withhout being swapped in, from my understanding). It sounds ideal for loading additional data into those areas, a bit like the old extended memory calls on MS-DOS, but perhaps not quite as easy to use for additional code areas as th ROM banking methods.
Re: Developing for MSX-DOS2 and >64kb
I had a tinker with this last night, but ran into the issue that the primary mapper couldn't give me any segments which makes me think I've misunderstood something: I presume the primary mapper is responsible for the onboard memory beyond 64k? Or did it create a RAM disc to use that memory up and I need to find a new emulator?
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
My understanding is that MSX-DOS2 required 128kb of RAM, and that it was primarily MSX2 machines that satisfy that criteria (and only a subset of those - since there are quite a few non-European models with only 64kb). I guess that's one of the problems with the MSX1|2|2+ being a loose collection of standards rather than a single manufacturer produced machine.
I don't know if the primary mapper on 128kb base machines (i.e. unexpanded via cart) can support segments above the built-in 128kb, but it should certainly be able to use them in the 64-128kb region. I don't know if MSX1 models have that functionality, but it should certainly be part of any MSX2 with 128kb from new.
On my (real) Sony HB-FX1DJ which has an internal expansion to 256KB, MSX-DOS2 (or, more accurately; Nextor - which I understand is DOS2 compliant) correctly detects all of that ram, without the use of an addon cart with it's own mapper (like my Megaflashrom, for example).... and it's definitely the built-in mapper doing that, rather than a cart. Plug in the MFR and it detects 768KB via the additional 512KB mapper in the cart.
I don't know if the primary mapper on 128kb base machines (i.e. unexpanded via cart) can support segments above the built-in 128kb, but it should certainly be able to use them in the 64-128kb region. I don't know if MSX1 models have that functionality, but it should certainly be part of any MSX2 with 128kb from new.
On my (real) Sony HB-FX1DJ which has an internal expansion to 256KB, MSX-DOS2 (or, more accurately; Nextor - which I understand is DOS2 compliant) correctly detects all of that ram, without the use of an addon cart with it's own mapper (like my Megaflashrom, for example).... and it's definitely the built-in mapper doing that, rather than a cart. Plug in the MFR and it detects 768KB via the additional 512KB mapper in the cart.
Re: Developing for MSX-DOS2 and >64kb
Ah right, if DOS2 requires 128kb, then I've got no free memory on a 128kb emulator. That matches with my observations at code and debug level.
So, I need to find a new emulator to go any further.
So, I need to find a new emulator to go any further.
Re: Developing for MSX-DOS2 and >64kb
In case you want to go further whilst I'm playing around with emulators I've attached the msxdos.asm I've been working on. It's got a lot of logging in so I can see where it breaks.
The basic idea is to allocate a segment, load the bank into it and keep track of the mapping between the z88dk bank id and the MSX segment. This mapping is then used in banked_call code.
appmake has been updated to write banks to disc, so the banked example should be good if you change the MSX cflags to -subtype=msxdos2
The basic idea is to allocate a segment, load the bank into it and keep track of the mapping between the z88dk bank id and the MSX segment. This mapping is then used in banked_call code.
appmake has been updated to write banks to disc, so the banked example should be good if you change the MSX cflags to -subtype=msxdos2
You do not have the required permissions to view the files attached to this post.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
I'd love to say I could look at this, alas I'm purely a C guy; the only assembly I have experience of was a m68k introductory course some 25 years ago at university... And it has long been pushed out of my brain by mundane trivia about which TV shows my kids like or how to operate the washing machine! :-D
It definitely sounds like you're making progress, however.
It definitely sounds like you're making progress, however.
Re: Developing for MSX-DOS2 and >64kb
Well, after a couple of tweaks it looks like it works:
The code does need a cleanup/tune etc, but I'm having trouble finding the combination of working emulated hardware and a debugger, so I'm leaving it as a working proof of concept.
The configuration provides a 32kb home page (from $100 to 32767) and 32768->49151 is banked.
I've pushed the change, to use it you'll need to compile with the flags: -subtype=msxdos2 -pragma-define:CRT_DISABLELOADER=0
The code does need a cleanup/tune etc, but I'm having trouble finding the combination of working emulated hardware and a debugger, so I'm leaving it as a working proof of concept.
The configuration provides a 32kb home page (from $100 to 32767) and 32768->49151 is banked.
I've pushed the change, to use it you'll need to compile with the flags: -subtype=msxdos2 -pragma-define:CRT_DISABLELOADER=0
You do not have the required permissions to view the files attached to this post.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
Wow! That was fast work. I'll pull down the latest version and recompile this weekend.
I'm putting the meat on the bones of my Sinclair QL target of my rpg engine, and am starting to design the MSX implementation on paper right now ... The extra capacity afforded by dedicated vram will allow me to do some proper animation and the banking support should let me swap in and out an audio player as well as extra effects that just wouldn't be possible in an unbanked implementation (and for me, with lots of on-disk assets, a ROM is not the way to go). This looks really promising.
I'm putting the meat on the bones of my Sinclair QL target of my rpg engine, and am starting to design the MSX implementation on paper right now ... The extra capacity afforded by dedicated vram will allow me to do some proper animation and the banking support should let me swap in and out an audio player as well as extra effects that just wouldn't be possible in an unbanked implementation (and for me, with lots of on-disk assets, a ROM is not the way to go). This looks really promising.
Re: Developing for MSX-DOS2 and >64kb
In theory a ROM could be up to 4MB in size which is the same size as I've allocated for the DOS banking.
However, we don't, and I've not come across, a portable ROM filesystem that supports compression
Let me know how it goes.
However, we don't, and I've not come across, a portable ROM filesystem that supports compression
Let me know how it goes.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
Okay, pulled down the latest commit and rebuilt this morning. However, like you, I'm struggling to find a combination of hardware models and extensions in OpenMSX that I can get this working on
On my real MSX2+, with (256kb onboard + 512kb Megaflashrom ram - including Nextor DOS), it doesn't get as far as printing the "Printing from main memory bank" message, the (real) floppy churns for a second or two after calling bank.com, but then hangs:
The memory arrangement on this machine is:
On my real MSX2+, with (256kb onboard + 512kb Megaflashrom ram - including Nextor DOS), it doesn't get as far as printing the "Printing from main memory bank" message, the (real) floppy churns for a second or two after calling bank.com, but then hangs:
The memory arrangement on this machine is:
You do not have the required permissions to view the files attached to this post.
Last edited by megatronuk on Sat Feb 05, 2022 10:31 am, edited 1 time in total.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
In OpenMSX I've tried the following:
Config 1. NMS-8255 (MSX2, 128kb onboard ram, added a Nextor 2.11 ASCII16 rom)
... hangs... and then after about 60 seconds prints a several screens worth of this:
Config 1. NMS-8255 (MSX2, 128kb onboard ram, added a Nextor 2.11 ASCII16 rom)
... hangs... and then after about 60 seconds prints a several screens worth of this:
You do not have the required permissions to view the files attached to this post.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
I also tried this config,
Config 2. NMS-8255 (MSX2, 128kb onboard ram, added the ASCII MSX-DOS2 cartridge.
But it seems to have the same behaviour as my real MSX2+, it just hangs indefinitely:
Config 2. NMS-8255 (MSX2, 128kb onboard ram, added the ASCII MSX-DOS2 cartridge.
But it seems to have the same behaviour as my real MSX2+, it just hangs indefinitely:
You do not have the required permissions to view the files attached to this post.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
Another result;
Config 3. OpenMSX, Sony HB-F1XDJ (64kb base, +512kb mapper option, +Nextor ASCII16 standalone rom)
This one actually does print the "Printing from main memory bank" message, but then does nothing more... a few seconds later OpenMSX shows the crash message seen in the screengrab.
Config 3. OpenMSX, Sony HB-F1XDJ (64kb base, +512kb mapper option, +Nextor ASCII16 standalone rom)
This one actually does print the "Printing from main memory bank" message, but then does nothing more... a few seconds later OpenMSX shows the crash message seen in the screengrab.
You do not have the required permissions to view the files attached to this post.
Re: Developing for MSX-DOS2 and >64kb
I’m not convinced that the disc layout is correct for openmsx which would explain what you’re seeing.
Without a working debugger it’s tricky to fix though.
Without a working debugger it’s tricky to fix though.
Re: Developing for MSX-DOS2 and >64kb
I should probably also restore the log messages to give a bit more clue as to what’s going on.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
Okay, I think I've got a working configuration for OpenMSX....
Sony MSX2+ HB-F1XDJ (64kb base + 512kb mapper + DOS2 rom (both the downloaded Nextor rom and the bundled msxdos2 extensions work))
Philips NMS-8255 (128kb base + NO mapper + DOS2 rom (both Nextor rom and the msxdos2 extensions work))
As expected, there aren't enough free banks remaining in a base 128kb configuration - although there's enough for one, it seems; that at least gets us 52kb + 16kb working memory, which is more than you would normally be able to get on a MSX-DOS2 system with the minimum amount of installed memory.
I'm still trying to get it to run on my real hardware... I am wondering if the Megaflashrom mapper is working differently to the 512kb mapper 'cart' that OpenMSX uses. I've tried replicating the Megaflashrom inside OpenMSX, but I can't get it to add it's own memory to that of the machine.
Sony MSX2+ HB-F1XDJ (64kb base + 512kb mapper + DOS2 rom (both the downloaded Nextor rom and the bundled msxdos2 extensions work))
Philips NMS-8255 (128kb base + NO mapper + DOS2 rom (both Nextor rom and the msxdos2 extensions work))
As expected, there aren't enough free banks remaining in a base 128kb configuration - although there's enough for one, it seems; that at least gets us 52kb + 16kb working memory, which is more than you would normally be able to get on a MSX-DOS2 system with the minimum amount of installed memory.
I'm still trying to get it to run on my real hardware... I am wondering if the Megaflashrom mapper is working differently to the 512kb mapper 'cart' that OpenMSX uses. I've tried replicating the Megaflashrom inside OpenMSX, but I can't get it to add it's own memory to that of the machine.
You do not have the required permissions to view the files attached to this post.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
BTW, the above is with copying bank.com and the bank files to a MSX-DOS2 formatted bootable floppy, not bank.img as created from the example directory.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
Okay, good news. The modified disk works on a real machine!
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
Booting from my Megaflashrom / hard drive image on SD card, and then loading the test banking code from a real formatted MSX-DOS2 floppy:
https://www.youtube.com/watch?v=7Ctat4j8sqQ
I don't follow the reason for the original failures - perhaps it was disk geometry or something weird like that, but that's now 3 for 3 on the two OpenMSX systems and this one real one.
https://www.youtube.com/watch?v=7Ctat4j8sqQ
I don't follow the reason for the original failures - perhaps it was disk geometry or something weird like that, but that's now 3 for 3 on the two OpenMSX systems and this one real one.
You do not have the required permissions to view the files attached to this post.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
I have expanded the banked example to 8 banks (which in theory should work on a MSX-DOS2 machine with 256kb or more - though possibly 192kb in practice), functions bank1 and bank2 are identical to the original code. bank3 through bank7 call the next bank in turn with a value, as bank3 did in the original code, e.g.
(and so on, in turn, so each function should print out the id of the function that called it in turn)
Then the final func_bank8 returns and should collapse the call stack back to main(), just like the original:
Each bank function gets passed the ID of the function that called it (func_bank4 gets called with the value 3, func_bank5 with 4, and so on)
It compiles, but instead of the func_bank3 then calling the function from bank4 and onwards, OpenMSX crashes with a 'DI' halt error when func_bank4 is (or is about to be) called:
Code: Select all
#define BANK 3
#pragma bank 3
#endif
int func_bank3(int value) {
printf("Printing from bank%d - passed value %d\n",BANK,value);
return(func_bank4(3));
}
Then the final func_bank8 returns and should collapse the call stack back to main(), just like the original:
Code: Select all
#if __CPC__
#define BANK 7
#pragma bank 7
#else
#define BANK 8
#pragma bank 8
#endif
int func_bank8(int value) {
// printf is in common code
printf("Printing from bank%d - passed value %d\n",BANK,value);
return(8);
}
It compiles, but instead of the func_bank3 then calling the function from bank4 and onwards, OpenMSX crashes with a 'DI' halt error when func_bank4 is (or is about to be) called:
You do not have the required permissions to view the files attached to this post.
-
- Member
- Posts: 34
- Joined: Fri Jan 28, 2022 11:39 pm
Re: Developing for MSX-DOS2 and >64kb
Also, an interesting difference in behaviour on a real system when attempting to use 8 banked functions as above - rather than hanging like OpenMSX does, it appears that bank.com terminates and simply returns to command2.com:
It's definitely terminating abnormally, as it doesn't return to main() to print the "Returned value from bank calls is..." string. It should be doing the nested call sequence all the way to func_bank8().
It's definitely terminating abnormally, as it doesn't return to main() to print the "Returned value from bank calls is..." string. It should be doing the nested call sequence all the way to func_bank8().
You do not have the required permissions to view the files attached to this post.
Re: Developing for MSX-DOS2 and >64kb
Using ASMLIB that's been built into Z88DK for a couple of years now banking in the MSX can be performed already. This banking also works on MSX1 computers if the MU.COM TSR has been loaded on MSXDOS1 (https://www.msx.org/downloads/mu-v105-w ... translated). ASMLIB hasn't really been documented much on the wiki however so I've converted it's documentation to markdown format and linked to it from the MSX platform page (https://github.com/z88dk/z88dk/wiki/Platform---MSX).
Here's some sample code of mine that does banking with ASMLIB:
I also have some assembler code that's run on startup that detects whether the memory banking extension is available (based on information gleaned from https://www.msx.org/wiki/Extended_Bios). This assembly code could be ported to C fairly easily with the bdos function :
code to detect the width and height of the console on MSXDOS 1 and 2:
Here's some sample code of mine that does banking with ASMLIB:
Code: Select all
#include <arch/z80.h> /* for asmlib */
#define ALL_SEG 0x00 /* Allocate a 16K segment */
#define FRE_SEG 0x03 /* Free a 16K segment */
#define RD_SEG 0x06 /* Read 1 byte from the segment specified by A at the address pointed to by HL */
#define WR_SEG 0x09 /* Write 1 byte from E to the segment specified by A at the address pointed to by HL */
#define CAL_SEG 0x0c /* Inter-segment call */
#define CALLS 0x0f /* Inter-segment call */
#define PUT_PH 0x12 /* Put a segment on the bank specified by HL */
#define GET_PH 0x15 /* Get the segment number of the bank specified by HL */
#define PUT_P0 0x18 /* Put a segment on bank 0 */
#define GET_P0 0x1b /* Get the segment number of bank 0 (0000h~3FFFh) */
#define PUT_P1 0x1e /* Put a segment on bank 1 */
#define GET_P1 0x21 /* Get the segment number of bank 1 (4000h~7FFFh) */
#define PUT_P2 0x24 /* Put a segment on bank 2 */
#define GET_P2 0x27 /* Get the segment number of bank 2 (8000h~CFFFh) */
#define PUT_P3 0x2a /* No effect */
#define GET_P3 0x2d /* Get the segment number of bank 3 (C000h~FFFFh) */
extern char hasMapper;
extern unsigned char loadPageStatus;
extern unsigned char defaultBank;
extern int mapperJumpTable;
extern Z80_registers regs;
void dosload(int pageNumber) __z88dk_fastcall {
const char * filename = "prefix00.ovl";
static int temp;
if(hasMapper) {
if(loadPageStatus == 0xff) {
/* no ram block has been allocated to this segment. try to allocate one now */
regs.Bytes.A = 0; /* allocate user segment */
regs.Bytes.B = 0; /* allocate from primary mapper */
AsmCall(mapperJumpTable + ALL_SEG, ®s, REGS_MAIN, REGS_AF);
regs.Bytes.B = 0;
if(regs.Flags.C) {
/* allocation failed. Switch to the default bank to load the code into that */
regs.Bytes.A = defaultBank;
}
else {
/* allocation suceeded. switch to the bank number that was returned */
loadPageStatus = regs.Bytes.A;
}
AsmCall(mapperJumpTable + PUT_P1, ®s, REGS_AF, REGS_NONE);
}
else {
/* a ram block has already been allocated to this segment. just switch to it and don't load anything */
regs.Bytes.A = loadPageStatus;
AsmCall(mapperJumpTable + PUT_P1, ®s, REGS_AF, REGS_NONE);
return;
}
}
sprintf(filename + 6, "%02d", pageNumber);
filename[8] = '.';
if((temp = open(filename, O_RDONLY, 0)) == -1) {
fputs_z80("Couldn't find ", stderr);
fputs_z80(filename, stderr);
exit(EXIT_FAILURE);
}
read(temp, (void *)16384, 16384);
close(temp);
}
Code: Select all
_initMapper: ; detect if a msx2 compatible mem mapper is present
; call CPM_VER. msx computers always return 0x22 but still implement more bdos calls than real cp/m 2.2 does
ld c, 0x0c
call 0x0005
cp 0x22
jr nz, noMapper
; test for whether this code is running on an MSX computer by calling MSX_DOSVER
ld a, 1
ld c, 0x6f
call 0x0005
or a
jr nz, noMapper
; test for presence of extended bios
ld a, (HOKVLD)
bit 0, a
jr z, noMapper ; no extended bios
; call GET_VARTAB to test for msx2 mapper support
xor a
ld de, 0x0401
call EXTBIOS
or a
jr z, noMapper ; no mapper support if a = 0
; call GET_JMPTAB and store the resultant address
ld de, 0x0402
call EXTBIOS
ld (_mapperJumpTable), hl
; store that the mapper was detected
ld a, 1
ld (_hasMapper), a
;store the default page segment number
ld de, 0x0021 ; get the segment number selected on second page (GET_P1)
xor a
add hl, de
call l_jphl
ld (_defaultBank), a
noMapper:
ret
Code: Select all
#include <cpm.h>
#define MSX_DOSVER 0x6f
...
if(
bdos(CPM_VERS, 0) == 0x22 && /* MSX computers return 2.2 as their CP/M version */
bdos(MSX_DOSVER, 0) == 0 && /* only MSX computers return 0 for this call */
/* MSX computers store the console size directly */
(width = *((unsigned char *)0xF3B0)) != 0 &&
(height = *((unsigned char *)0xF3B1)) != 0
) {isMSX = 1; }
Re: Developing for MSX-DOS2 and >64kb
Whoops. I forgot to include definitions for EXTBIOS and HOKVLD:
Code: Select all
EXTBIOS equ 0xFFCA
HOKVLD equ 0xFB20