Arranging segments in memory

ZX80, ZX 81, ZX Spectrum, TS2068 and other clones
Post Reply
w00tguy
New member
Posts: 9
Joined: Thu Aug 24, 2017 11:38 pm

Arranging segments in memory

Post by w00tguy »

I want to choose where the code/data/bss/stack are in memory, but I'm running into problems:

1) -pragma-define:CRT_ORG_DATA=[address] doesn't seem to work. No matter what I set this to I don't see any difference in the memory viewer (using the Fuse emulator). Is this not supported?

2) -pragma-define:CRT_ORG_BSS=[address] works, but the hello world program glitches out after moving it somewhere else (tried a few locations). Possibly related to #1?

3) There's about 64 bytes of data placed directly before the CODE area in the loaded program. I have no idea what this is - the byte sequence doesn't appear in the compiled binary or .tap file. I thought it might be the data segment, but I can memset the area and the program keeps working (infinite printf loop). I've already moved the stack, too. Any idea what this is?
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

w00tguy wrote:I want to choose where the code/data/bss/stack are in memory, but I'm running into problems:

1) -pragma-define:CRT_ORG_DATA=[address] doesn't seem to work. No matter what I set this to I don't see any difference in the memory viewer (using the Fuse emulator). Is this not supported?

2) -pragma-define:CRT_ORG_BSS=[address] works, but the hello world program glitches out after moving it somewhere else (tried a few locations). Possibly related to #1?
The newlib gives you control over everything (it was designed with sections in mind from the beginning), the classic lib gives you some control over CODE & BSS (DATA will always append to either CODE or BSS).

There are three compile models:

0. ram :- the program is destined for ram. The code is sequenced as CODE+DATA/BSS. You can move BSS independently of CODE+DATA.

1. rom :- the program is destined for rom (but can work for ram too). The code is sequenced as CODE/BSS+DATA. The BSS+DATA location is controlled by the BSS org.

2. compressed rom :- the program is destined for rom (the stored data section is compressed, can work for ram too). As for 1, the code is sequenced as CODE/BSS+DATA. The BSS+DATA location is controlled by the BSS org.

So CRT_ORG_CODE will control the code origin, CRT_ORG_BSS will control the bss origin and the DATA section will always append to CODE or BSS depending on the compile model.

The default compile model for classic zx is the ram model. You can change that by defining CRT_MODEL. For the classic zx, the initial stack pointer is controlled by STACKPTR.

(For reference the newlib can place CODE/DATA/BSS independently with BSS normally appending to DATA and stack location is controlled by REGISTER_SP).
3) There's about 64 bytes of data placed directly before the CODE area in the loaded program. I have no idea what this is - the byte sequence doesn't appear in the compiled binary or .tap file. I thought it might be the data segment, but I can memset the area and the program keeps working (infinite printf loop). I've already moved the stack, too. Any idea what this is?
Are these 64 bytes part of the stack? I'm guessing it's space declared on the stack to register atexit functions. The standard requires 32 slots.
The newlib allows the size to be set via CLIB_EXIT_STACK_SIZE (0 by default).
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

I tried moving bss and code for the hello program and it seems to be working for me.

You can add "-m" to the compile line to get a memory map and that way check that things are actually moved. You must be careful with where the stack is located and, if you use malloc, where the heap is located.
w00tguy
New member
Posts: 9
Joined: Thu Aug 24, 2017 11:38 pm

Post by w00tguy »

Thanks for the info - I didn't know there were different flags for each lib. Switching to the new lib solved the BSS glitch problem, but I still can't move the DATA segment. These are the flags I'm testing with:

Code: Select all

+zx -clib=new -startup=1 -m \
-pragma-define:CRT_ORG_CODE=0x8000 \
-pragma-define:CRT_ORG_DATA=0xC000 \
-pragma-define:CRT_ORG_BSS=0 \
-pragma-define:CLIB_MALLOC_HEAP_SIZE=0
According to these docs, that should move the DATA & BSS segments both into the 0xC000 area. I can see in the .map file that __DATA_head and __BSS_head are pointing to the right place, but the program prints some random text and hangs when I do this. Moving just BSS and not DATA works fine.
alvin wrote:Are these 64 bytes part of the stack? I'm guessing it's space declared on the stack to register atexit functions. The standard requires 32 slots.
The newlib allows the size to be set via CLIB_EXIT_STACK_SIZE (0 by default).
After moving the stack with REGISTER_SP the data was still there. It probably is some other sort of stack, but changing CLIB_EXIT_STACK_SIZE didn't affect it.
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

w00tguy wrote:Thanks for the info - I didn't know there were different flags for each lib. Switching to the new lib solved the BSS glitch problem, but I still can't move the DATA segment. These are the flags I'm testing with:

Code: Select all

+zx -clib=new -startup=1 -m \
-pragma-define:CRT_ORG_CODE=0x8000 \
-pragma-define:CRT_ORG_DATA=0xC000 \
-pragma-define:CRT_ORG_BSS=0 \
-pragma-define:CLIB_MALLOC_HEAP_SIZE=0
According to these docs, that should move the DATA & BSS segments both into the 0xC000 area. I can see in the .map file that __DATA_head and __BSS_head are pointing to the right place, but the program prints some random text and hangs when I do this. Moving just BSS and not DATA works fine.
I know what it is.. the tap file generator is not including the DATA section in the load. Just to verify this, do a compressed rom model compile by adding "-pragma-define:CRT_MODEL=2" to the compile line.

Moving DATA & BSS up to 0xc000 is a little unusual. Are you trying to use the 128k's extra banks? Usually code/data/bss would be considered part of the "main binary" that exists in the non-banked part of the program. You can assign code and data to the 128's banks by assigned them to sections BANK_NN where NN=00-07 (both asm and C can be placed in those banks). After that you can take control over which parts of DATA and BSS get moved to other memory banks by defining your own memory map.

It's getting late here so I'll have more to say tomorrow.
After moving the stack with REGISTER_SP the data was still there. It probably is some other sort of stack, but changing CLIB_EXIT_STACK_SIZE didn't affect it.
I think it's probably the basic system then. If you're producing a tap file, the loader z88dk provides will "CLEAR n-1" where n = your code ORG. This will get basic to move its stuff just under address n.
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

The docs you are referring to apply to the newlib. The classic lib has some similar functionality for placement of DATA & BSS as described in post #2.

If you have a lot of pragmas it's easier to consolidate them into a single file:

zpragma.inc

Code: Select all

#pragma output CRT_ORG_CODE = 0x8000   // move code origin
#pragma output CRT_ORG_DATA = 0xC000   // move data origin
#pragma output CRT_ORG_BSS = 0    // bss appends to data

#pragma output CLIB_MALLOC_HEAP_SIZE = 0   // no heap
A simple test compile of hello world:

hello.c

Code: Select all

#include <stdio.h>

void main(void)
{
   printf("Hello World!\n");
}
zcc +zx -vn -startup=30 -clib=sdcc_iy hello.c -o hello -pragma-include:zpragma.inc -create-app

In the output you will see files "hello_CODE.bin" and "hello_DATA.bin". This will correspond to the code section and the data+bss section (the binary is named after the section with a real org). You should also be able to see your pragmas in the generated file "zcc_opt.def".

The pragmas and the compile are always working.

The problem comes from appmake, the application maker that is called when "-create-app" appears on the compile line. It will only use "hello_CODE.bin" to make the tap/sna/etc file. So you're going to get a tap file that only loads the code part and not the data part and that will lead to a non-functioning program.

If you switch to a compressed rom model compile by adding the pragma "CRT_MODEL = 2", then the tools will output three files "hello_CODE.bin", "hello_DATA.bin" and "hello_BSS.bin". The pragmas are still being obeyed - the bss section will follow the data section in memory. However, rom model compiles initialize ram (both DATA and BSS) so the CODE part will include a ram initializer (CODE + compressed stored DATA + decompressor). So appmake will make a tap out of the CODE+ram_intitializer and the program will run properly because at startup the crt will initialize DATA+BSS.

Going back to your ram model compile (output is "hello_CODE.bin" and "hello_DATA.bin"), the problem is the DATA + BSS sections are not being loaded by the tap created by appmake. You can get around this by making your own loader:

zcc +zx -vn -startup=30 -clib=sdcc_iy hello.c -o hello -pragma-include:zpragma.inc

(no create-app, don't need an incorrect tap as output)

Code: Select all

10 CLEAR 32767: REM CRT_ORG_CODE - 1
20 LOAD ""CODE: REM CODE SECTION
30 LOAD ""CODE: REM DATA SECTION
40 RANDOMIZE USR 32768
Save that to a tap with SAVE "loader" LINE 1. Maybe name the tap "loader.tap". You can use an emulator to make the tap file or a command line utility like bas2tap.

Use appmake directly to make taps out of the files "hello_CODE.bin" and "hello_DATA.bin" :

Code: Select all

appmake +zx -b hello_CODE.bin --org 0x8000 -o code.tap --noloader --blockname code
appmake +zx -b hello_DATA.bin --org 0xc000 -o data.tap --noloader --blockname data
Then put the whole thing together in one tap file:

Code: Select all

(windows)  copy /b loader.tap+code.tap+data.tap program.tap
(not windows)   cat loader.tap code.tap data.tap > program.tap
Then "program.tap" will contain a working program as expected.
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

However, rom model compiles initialize ram (both DATA and BSS) so the CODE part will include a ram initializer (CODE + compressed stored DATA + decompressor). So appmake will make a tap out of the CODE+ram_intitializer and the program will run properly because at startup the crt will initialize DATA+BSS.
Actually the CODE file emitted will not contain the compressed DATA + decompressor but appmake will form an internal file containing CODE + compressed stored DATA + decompressor that will be used to make the tap file.
w00tguy
New member
Posts: 9
Joined: Thu Aug 24, 2017 11:38 pm

Post by w00tguy »

That's a lot of helpful info, thanks! You were right that setting CRT_MODEL=2 fixed the problem. Creating the custom tape loader also worked, and that's what I'm using now. I assume I would need to do this anyway since I want to have more code/data loaded from tape after the program starts.
w00tguy
New member
Posts: 9
Joined: Thu Aug 24, 2017 11:38 pm

Post by w00tguy »

alvin wrote:I think it's probably the basic system then. If you're producing a tap file, the loader z88dk provides will "CLEAR n-1" where n = your code ORG. This will get basic to move its stuff just under address n.
Yep, I found some more info on this:
http://www.worldofspectrum.org/ZXBasicM ... hap24.html
It's for the GOSUB stack in BASIC. I can overwrite that area since I don't plan to return to BASIC from my program.

There's also some data at the end of memory (0xFF00-0xFFFF in the 48k) that appears to be more BASIC data (previous RAMTOP + UDG?), but that area also appears to be used by the z88dk libraries. The area is written to when I use printf(), so I guess I shouldn't be touching it.
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

w00tguy wrote:There's also some data at the end of memory (0xFF00-0xFFFF in the 48k) that appears to be more BASIC data (previous RAMTOP + UDG?), but that area also appears to be used by the z88dk libraries. The area is written to when I use printf(), so I guess I shouldn't be touching it.
That looks like the stack - the default stack location is 65368 for compatibility with basic.

The default crt config for a spectrum ram compile is in this file:

https://github.com/z88dk/z88dk/blob/mas ... ig.inc#L19

There you can see REGISTER_SP=65368 which can be changed with a pragma.

The options are briefly explained here:

https://www.z88dk.org/wiki/doku.php?id= ... figuration

There are things you may want to do:

* Change exit behaviour if you're not returning to basic (CRT_ON_EXIT). If you're not returning to basic then the crt doesn't have to include code that makes that possible.
* Eliminate the heap if you're not using it (CLIB_MALLOC_HEAP_SIZE) or alter its size and location if you will be using it. CRT_STACK_SIZE only matters if CLIB_MALLOC_HEAP_SIZE=-1. In this arrangement the heap extends from the end of your program up to the stack and the CRT_STACK_SIZE variable determines how much stack space there will be.
* Eliminate stdio's heap if you won't be opening memstreams; this pool of memory is actually for creating FILE structures and file descriptors but that isn't possible yet for anything except memstreams. (CLIB_STDIO_HEAP_SIZE).
w00tguy
New member
Posts: 9
Joined: Thu Aug 24, 2017 11:38 pm

Post by w00tguy »

You're right, it's the stack. The stack was placed before the code area when I was using the classic lib and I didn't consider that it had a new default.

Thanks for the tips, I've updated my pragmas.
Post Reply