Memory banking question

ZX80, ZX 81, ZX Spectrum, TS2068 and other clones
Post Reply
Stefan123
Member
Posts: 85
Joined: Fri Oct 21, 2016 7:57 am

Memory banking question

Post by Stefan123 »

I have a Spectrum Next program located in the address range 0x8000 to 0xFFFF. When the program starts, I want to play some intro music. However, I don't want to include the sound driver in the main program since it's only used initially and I don't want to waste precious memory. I was thinking of using the memory between 0x4000 and 0x8000 for temporarily loading the sound driver and the sound data (the ULA screen is not used at this point). What is the best way to accomplish this?

My first idea was to create a separate binary that contains the sound driver and sound data and ORG it to address 0x4000. Then I would load this binary file into memory at address 0x4000 using esxdos and have a trampoline function in the main program that calls the entrypoint in the sound driver.

Another idea is to somehow use the memory banking support in z88dk but I don't know how that works. I assume that z88dk somehow can create a separate bank file for a specific piece of code and then you can page in that memory bank to, for example, address 0x4000. Does the CRT automatically put these bank files in their designated pages? How do I make sure that the code in a bank is executable at the address I want to page it in to? Maybe I'm totally off track here?

Any feedback is much appreciated :)
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

There are several ways to do what you want.

1.

If you want to load from disk on demand such that the code does not occupy any memory pages at load time, maybe it makes most sense to just create a custom section with org to hold it so that the linker spits out a separate binary for the section.

The most general way to do this is to set "CRT_APPEND_MMAP = 1" in a pragma. This informs the crt that you want to add sections to the memory map from a file "mmap.inc". In this file, add:

SECTION INTRO
org 0x4000

To put stuff there, you just assign to section INTRO. If it's asm then you add "SECTION INTRO" before the asm. If it's C then you change the section assignment away from the default "-codesegINTRO -constsegINTRO" (this will affect CODE and RODATA respectively; sdcc cannot do BSS/DATA currently but sccz80 can) ; this means for C you must separate your c into files that go the same memory banks because the bank assignment applies to all c files listed. So normally you compile to object file using the bank assignment and then add the object files to the final compile that makes the binary.

2.
You can assign code/data to a specific memory bank or page with any org address you want. The bank or page number tells where in the Next's memory that something is placed. So the 2MB next has 16k banks 0-111 (0-7 correspond to the 128k spectrum) or, named another way, 8k pages 0-223. Separately an org can be set. If you decide to place something in page 100 with org 0x4000 then that thing will be placed at offset 0x4000&0x1ffff = 0 in page 100. To run it you must place page 100 in mmu 2 (the 0x4000-0x5ffff slot) because it's only there that the org is valid. Suppose you org at 0x5000 instead. Then that thing will be placed at offset 0x5000&0x1fff = 0x1000 in page 100. When you place that at mmu2, the offset of 0x1000 ensures that that thing is at address 0x4000+0x1000=0x5000.

There are three ways that the same memory is named:

BANK_n where n is a decimal number 0-111. This is naming the memory in 16k chunks like on the 128k spectrum.
BANK_n_L this names the lower 8k part of the 16k bank n.
BANK_n_H this names the upper 8k part of the 16k bank n.

PAGE_n where n is a decimal number 0-223. This names memory in 8k chunks and is the native Next banking method using mmus. BANK m occupies two 8k pages: PAGE m*2 and PAGE m*2+1

You can choose to use any of these sections to place things. The BANK sections enforce 16k size restrictions. If anything you put in there exceeds the 16k boundary, you'll get an error. The BANK_n_L and BANK_n_H enforce 8k sizes.

PAGE_n has no size enforcement. If you assign to PAGE_m and your stuff exceeds PAGE_m, it just spills into the next page.

The crts define default ORG for each of these sections (either 0xc000 or 0xe000) which you can change with pragmas. But actually what you will be doing most of the time is defining your own sections that go into these banks or pages. If the name of the section contains "BANK_n", "BANK_n_L", "BANK_n_H" or "PAGE_n" as a substring, then that section goes into the correspoding bank or page with the org address you supply.

It works the same way as in 1. Suppose you want to put something into page 100 with org 0x5000. Then do the CRT_APPEND_MAP=1 pragma and in the mmap.inc file:

SECTION PAGE_100_INTRO
org 0x5000

You can add as many sections as you want to mmap.inc.

The build tools will automatically merge everything together and identify problems like memory overlap.



What does appmake do with this?

-subtype=tap. This is from the spectrum and is not aware about memory in any way. You will get a bunch of unprocessed binaries, not recommended.

-subtype=sna. The 128k banks 0-7 (pages 0-15) are placed in the sna with assumption the program starts with the usual 16k banks 5,2,0 paged in. Separate binaries will be produced for each 16k bank occupied (BANK_N.bin). You may want to add the options -Cz"--clean --fullsize --filler 0". The first one will delete binary files used to make the sna, the last two will make sure all produced bank binaries are 16k in size and spaces filled with 0s.

-subtype=bin. Similar to sna except all banks, including the "main bank" are produced in separate binary files.

-subtype=zxn. Not done yet - this one will form a monolithic binary to be saved to sd card that will automatically load itself into all banks and run.
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

-subtype=dot
-subtype=dotx
-subtype=dotn

Like other subtypes, extra banks are output as separate binaries.

-subtype=dotn deserves special mention. It allocates and deallocates 8k pages from NextOS so that it can co-exist with anything running on the system. There is a pragma that allows you to request extra pages which are then made available to the program through an array.
Stefan123
Member
Posts: 85
Joined: Fri Oct 21, 2016 7:57 am

Post by Stefan123 »

Thanks for all the information, that was a lot to digest :) I think I need to try the different options in practice with a test program to understand them better.

The forthcoming subtype=zxn option sounds really good. Would that support embedding resource files like graphics and sound in the monolithic binary?
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

Stefan123 wrote:The forthcoming subtype=zxn option sounds really good. Would that support embedding resource files like graphics and sound in the monolithic binary?
Yes I hope so. I think it will end up having a header containing several command blocks, like load x bytes to page n, and perhaps other types as well. Maybe initializing the next hw to what is expected by the program could be in a block too. For resources, what I am thinking about is passing to the program the name of its file and an offset into the file to the first byte not loaded into memory, ie resources. Then the program can seek relative to that point to load resources as needed. Maybe a table of resources and file offsets could be generated too so that the program can easily find things.
Stefan123
Member
Posts: 85
Joined: Fri Oct 21, 2016 7:57 am

Post by Stefan123 »

I like the idea of a table of embedded resource files and their offset in the monolithic binary. Maybe having their file size as well in this table or would fstat() be supported for this type of embedded resources?
Stefan123
Member
Posts: 85
Joined: Fri Oct 21, 2016 7:57 am

Post by Stefan123 »

I tried the first option in a simple test program but it doesn't work the way I hoped, most likely I have missed some vital detail. My test program is available here: sound_test.zip. It uses my C API for the Vortex Tracker II player at https://github.com/stefanbylund/vt_sound but with all SECTION declarations changed to "SECTION INTRO".

I was expecting that z88dk would create a file called sound_test_INTRO.bin (containing the code/data tagged with the INTRO section) whose symbols would start at address 0x4000 and the references to those symbols in the main binary would match these addresses. What I actually got is a file called sound_test_UNASSIGNED.bin whose symbols start at address 0. The following warning is emitted: "Warning: Non-empty UNASSIGNED section ignored - this indicates that some code/data is not part of the memory map".
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

Your compile line is specifying two pragma files. There can only be one in a compile line so it takes the last one "mmap.inc" which is not a pragma file at all.

Correction:

zcc +zxn -subtype=sna -vn -O3 -startup=30 -clib=new -m vt_sound.asm PT3PROM.asm sound_test.c sound_data.asm -o sound_test -pragma-include:zpragma.inc -create-app

This one will produce the separate INTRO binary, however appmake thinks the section belongs in the "main binary" because it does not have a name leading in "BANK_" or "PAGE_". So that means it will place it at address 0x4000 in the sna snapshot.

You can confirm that if you add -Cz--clean because this option causes appmake to erase all binary files it uses to make the sna. When that is there the separate INTO binary is deleted.

Since you want appmake to ignore the INTRO section, you can say so with the exclude-sections option:

zcc +zxn -subtype=sna -vn -O3 -startup=30 -clib=new -m vt_sound.asm PT3PROM.asm sound_test.c sound_data.asm -o sound_test -pragma-include:zpragma.inc -Cz"--clean --exclude-sections INTRO" -create-app

This time, despite clean being there, the INTRO binary will remain and you can copy that to your sd card for loading at runtime.
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

I just tried it out in cspect and it seems to work!
Stefan123
Member
Posts: 85
Joined: Fri Oct 21, 2016 7:57 am

Post by Stefan123 »

Thanks for a second pair of eyes! I totally missed the double pragma includes...

I also ran the code and it really works, the random graphics almost seems in line with the music :)

Out of curiosity, when you create a custom section like this, will the z88dk linker internally structure the content of the section to contain the code, data and bss in that order or are they intermingled?

If you have a precompiled object file (from an assembly source file) declared to be in some section, is it possible to override the section of that object file to be something else by passing options to the z88dk linker?
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

> Out of curiosity, when you create a custom section like this, will the z88dk linker internally structure the content of the section to contain the code, data and bss in that order or are they intermingled?

No the separation done in the main binary is accomplished by having different sections defined, eg code_compiler, data_compiler, bss_compiler, etc. And then the compiler places stuff in the relevant section. Within any single section, the order depends on the order the linker sees the object files and libraries as the binary is built (ie it's 'undefined').

But you can achieve the same thing if you want by defining a few more sections and then assigning to the right one as appropriate. Right now you have this in mmap.inc:

SECTION INTRO
org 0x4000

But you can stratify the output INTRO binary by defining more sections:

SECTION INTRO
org 0x4000
section code_intro
section data_intro
section bss_intro

Output binaries are only generated for sections with org so again the output will be one binary *_INTRO.bin. The sections without org that follow are merged with section INTRO (the last section with an org). So anything assigned to section INTRO itself will appear first, followed by anything assigned to section code_intro, followed by data_intro and lastly bss_intro.

So if you want that separation you can do it by assigning to the relevant sections.

> If you have a precompiled object file (from an assembly source file) declared to be in some section, is it possible to override the section of that object file to be something else by passing options to the z88dk linker?

Not at the moment but it's something that should be doable. It's probably pretty easy to have a tool that can edit an object file or library to change section assignment.
Post Reply