Compiling a ROM replacement with custom CRT0 - how to handle data/bss

ZX80, ZX 81, ZX Spectrum, TS2068 and other clones
Post Reply
periata
New member
Posts: 6
Joined: Sat May 05, 2018 10:52 am

Compiling a ROM replacement with custom CRT0 - how to handle data/bss

Post by periata »

I'm trying to write a complete ROM replacement (a language interpreter) and how good the end result is going to be is likely to depend on how much stuff I can fit into the ROM, so want to keep everything as compact as possible. For that reason, I've started by implementing a CRT0 replacement (as the standard one wastes space between the RST vectors and before the NMI vector that I can cram useful stuff into). But I can't figure out how to make data and BSS sections work. I don't expect to have much preallocated writeable data (most of my data is likely to be read only), so I don't think I need the compressed data support, but the questions I have are:

1. How do I tell the compiler that DATA + BSS will be grouped together and relocated to 5B00h?
2. How do I find the copy of the initialized data section and its length in order to move them?
3. I currently get a warning about "myzorg" not being defined, and defaulting to 0 -- obviously this is the value I actually need, but how do I stop the warning?

Currently I'm compiling with the following settings:

Code: Select all

LIB=-lm
APPMAKE_OPTS=-Cz"--org 0 --rombase=0 --romsize=0x4000"

zcc +zx -subtype=rom -c -O3 ...
zcc +zx -subtype=rom -zorg=0 -o utalk-vm.rom -create-app $(APPMAKE_OPTS) -crt0=$(VMINIT) $(VMOBJ) $(LIB)
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

In the above you are using the classic library. The newlib library has a +z80 target that is intended for bare metal machines that you may want to check out.

1.
The documentation is here:
https://www.z88dk.org/wiki/doku.php?id= ... t_embedded

2.
Newlib source is rooted here:
https://github.com/z88dk/z88dk/tree/mas ... EVELOPMENT

The +z80 target is defined in a subdirectory:
https://github.com/z88dk/z88dk/tree/mas ... target/z80

3.
The crt for +z80 is here:
https://github.com/z88dk/z88dk/blob/mas ... t_0.asm.m4

The includes you see in that crt are rooted in ../target/z80:
https://github.com/z88dk/z88dk/tree/mas ... target/z80

so most include files are coming from:
https://github.com/z88dk/z88dk/tree/mas ... ENT/target

What goes into the crt is controlled by pragmas as described in the documentation link earlier:
https://www.z88dk.org/wiki/doku.php?id= ... tomization

4.
The default page zero code is here:
https://github.com/z88dk/z88dk/blob/mas ... ro_z80.inc

It attempts to fill in the gaps with code that is likely to be used by the library.
The default page zero will be added if the org is set to 0 (it is by default) "#pragma output CRT_ORG_CODE = 0" and the user hasn't indicated he wants to replace it "#pragma output CRT_INCLUDE_PREAMBLE = 1" (this includes code directly in front of the crt cod and will replace the page zero code when org is 0).


.. another post on custom crts.
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

It turns out I'm out the door.. I'll come back a little later on for info on using custom crts if someone else doesn't. There's a pragma inside the newlib that will replace the crt with yours. If that happens, you're expected to supply the memory map which determines where stuff like BSS and DATA is placed. You can have a look at how the newlib initializes BSS and DATA by looking at "clib_init_bss.inc" and "clib_init_data.inc" in https://github.com/z88dk/z88dk/tree/mas ... ENT/target . It grabs the end points of those areas as defined in the memory map.
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

The newlib default memory map is here:
https://github.com/z88dk/z88dk/blob/mas ... el_z80.inc

This gets included into the crt - the memory map must be in the first file seen by the linker. If using the newlib you can use this one or modify this one; the newlib library defines many sections to give the user more ability to place things in different banks. The classic library has far fewer sections.
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

This post will show how to make a fully custom build using the +z80 target in the newlib. The other course is to generate a new target in newlib that would be tailored to your hw.

1.
Create a "zpragma.inc" file to hold these pragmas. The pragmas get written to a file "zcc_opt.def" during compilation and these are defines read by the library-supplied crt at assemble time. Some of these defines will tell the library crt to use your crt instead.

zpragma.inc

Code: Select all

// use my crt in file "crt.asm.m4"

#pragma output startup = -1

// use my crt configuration in file "crt_cfg.inc"

#pragma output __CRTCFG = -1

// use my memory map in file "mmap.inc"

#pragma output __MMAP = -1   ;; I suggest keeping the library's by setting to 0

// no heaps

#pragma output CLIB_MALLOC_HEAP_SIZE = 0
#pragma output CLIB_STDIO_HEAP_SIZE = 0
This pragma file will be provided on the zcc compile line. The main crt file "z80_crt.asm.m4" will read it:
https://github.com/z88dk/z88dk/blob/mas ... asm.m4#L21

Because "__STARTUP=-1" (this comes from "startup" for legacy reasons), your crt file will replace the one supplied by the library.
Also there are a couple of constants defining a default memory map (__MMAP) and a default crt configuration (__CRTCFG). If you use the pragmas above, these will be set to -1 which indicate that you will be providing these files as well. You may actually want to use the default __MMAP=2 at least.

At this point your crt is all there is. It is up to you what is in the crt and whether you pay attention to CRTCFG or MMAP. But what you do have to do at minimum is set up a memory map.

You can use the library's crt as guide:
https://github.com/z88dk/z88dk/blob/mas ... t_0.asm.m4

I would start your crt almost the same way:

Code: Select all

include(`z88dk.m4')      ;;; include some m4 macro definitions in z88dk/src/m4 which does nothing if you don't use them

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;                   z80 standalone target                   ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; GLOBAL SYMBOLS ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

include "config_z80_public.inc"    ;;; machine generated defines from target configuration that are exported to the program

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; CRT AND CLIB CONFIGURATION ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

include "../crt_defaults.inc"    ;; z88dk crt configuration defaults
include "crt_config.inc"        ;; +z80 crt configuration defaults override z88dk ones
include(`../crt_rules.inc')    ;; rules determine final value of crt configuration
include(`z80_rules.inc')    ;; more crt configuration for the +z80 target only

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; SET UP MEMORY MAP ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

include(`crt_memory_map.inc')   ;; define the memory map

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; INSTANTIATE DRIVERS ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

; The embedded target has no device drivers so it cannot
; instantiate FILEs.

; It can use sprint/sscanf + family and it can create
; memstreams in the default configuration.

;;; cost of an empty driver instantiation section is about 6 bytes.  It will allow
;;; the sscanf and sprintf families to run as these generate a fake FILE structure
;;; and this fake FILE structure is tested for validity before running.  The validity
;;; test requires those 6 bytes.  The validity test is a library build time option that
;;; could be disabled if you rebuild the z80 library and then this section could
;;; be eliminated completely.  However, should you ever want to add any streams
;;; this is where they go.

include(`../clib_instantiate_begin.m4')

ifelse(eval(M4__CRT_INCLUDE_DRIVER_INSTANTIATION == 0), 1,,
`
   include(`crt_driver_instantiation.asm.m4')
')

include(`../clib_instantiate_end.m4')

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; STARTUP ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

SECTION CODE

PUBLIC __Start, __Exit

EXTERN _main

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; PAGE ZERO ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

IF (ASMPC = 0) && (__crt_org_code = 0)

   ;;;;;include "../crt_page_zero_z80.inc"

   ;;; include or place your page zero code here
   ;;; fall through or jump to __Start

ENDIF

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; CRT INIT ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

__Start:

.....
;;;; select what you want from the library crt or rewrite yourself.  The library crt
;;;; ensures things like the exit stack can function and offers options for what is
;;;; done on program exit.
The home directory for the includes is z88dk/libsrc/_DEVELOPMENT/target/z80/
https://github.com/z88dk/z88dk/tree/mas ... target/z80

You can see what the crt is doing and how by following the includes.



Your crt is a .asm.m4 file which means it is processed by m4 first before being assembled. You can ignore m4 and just use asm if you want. I left some comments in the above detailing what the purpose of some parts is.

The "CRT AND CLIB CONFIGURATION" section determines the final value of pragmas. Some of these are used by appmake to form the output rom or binary. Having these here will solve the "not compiled by z88dk" problem. Appmake looks at the org address in these pragmas for information while generating the output. It is up to your crt to implement the options defined by these pragmas or not (see https://www.z88dk.org/wiki/doku.php?id= ... figuration ). In the zpragma.inc file, specifying "__CRTCFG = -1" means you will be supplying a crt configuration file which this bit will use to generate the final crt configuration values. You'll want to change or copy one of the configs supplied by the library: https://github.com/z88dk/z88dk/blob/mas ... g.inc#L117

The memory map is the other important item. It tells the linker where to put code and variables. I'd strongly suggest using MMAP=0 so the library supplies the memory map:
https://github.com/z88dk/z88dk/blob/mas ... ap.inc#L22

This includes the default library memory map:
https://github.com/z88dk/z88dk/blob/mas ... el_z80.inc

There are a lot of sections defined in there for modules in the library so that a custom memory map would have fine control over where the library code is placed. However this memory map essentially defines three regions: CRT+CODE, DATA, BSS. code or data are assigned to sections by name which are just containers that hold bytes. This memory map sequences those containers in memory. sections without an org are appended to the section above them. A section with org is output as an independent binary. So you can see in this memory map there are only three sections that might have an org: CODE,DATA,BSS. The output from the linker will be up to three binaries: *_CODE.bin, *_DATA.bin and *_BSS.bin which will hold the contents of these master sections. Then appmake will use this and information in the map file to make the final output in the form requested.

The other feature of this memory map is how it sequences the crt itself:

section code_crt_init
section code_crt_main
section code_crt_exit
section code_crt_return
section code_crt_common

If you look at the library's crt you'll see these sections mentioned. "section code_crt_init" is a point where outside code (library or user) can insert initialization code before main is called. "section code_crt_exit" is a point where outside code can insert exit code run before files are closed.

Other sections allow placement of an im2 vector table inside page zero or elsewhere, and so on.

You can create your own memory map but I think it's much easier to just use this one unless you have some radically different needs.

One last thing. You can see how the library's bss and data section initialization is performed here:

https://github.com/z88dk/z88dk/blob/mas ... it_bss.inc
https://github.com/z88dk/z88dk/blob/mas ... t_data.inc

They grab end points defined in the memory map to know how big these sections are and where they are in ram. For rom compiles, it is expected that either a plain copy or a compressed copy of the data section is appended to the code stored in rom so that the data section contents can be copied from there to ram. The code stored in rom is the *_CODE.bin file. So you'd append either *_DATA.bin or a compressed version of *_DATA.bin to that file to form what goes in rom. This is what -create-app will do for you automatically.


******

Anyway, that should be enough to get going. But I would encourage you to look at the +z80 target as-is because it has a great deal of flexibility built in to the supplied crt that can likely do what you want already.
periata
New member
Posts: 6
Joined: Sat May 05, 2018 10:52 am

Post by periata »

Thanks to all the above suggestions I've now got a build working and a quick demo running on an emulator, but I'm trying to reduce the size a little further, as there is a lot of library code I don't need being included right now, include a whole load of stdio-related code, which I definitely don't want.

Following advice at https://www.z88dk.org/forum/viewtopic.php?pid=14832 I tried using

Code: Select all

#pragma output CLIB_FOPEN_MAX = -1
to remove the open file list, but this is giving me a build error:

Code: Select all

Errors in source file /usr/local/share/z88dk//libsrc/_DEVELOPMENT/target/z80/z80_crt.asm:
Error at file 'stdio/z80/asm__fflushall_unlocked.asm' line 37: symbol '__stdio_open_file_list' not defined
How can I successfully remove all the stdio related code?
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

stdio code will not be brought into the compile unless you're using it.

Do you have a compile line we can see?

Parts of stdio can be brought in if you use a crt that instantiates drivers on stdin/stdout/stderr, if you use the sprintf/sscanf family, if you explicitly select a subset of printf or scanf comverters using "#pragma printf = ..." or "#pragma scanf = ..."
periata
New member
Posts: 6
Joined: Sat May 05, 2018 10:52 am

Post by periata »

Ah, I was setting up sprintf ... I had assumed that could be used without the rest of stdio. OK. Never mind. :)
alvin
Well known member
Posts: 1872
Joined: Mon Jul 16, 2007 7:39 pm

Post by alvin »

periata wrote:Ah, I was setting up sprintf ... I had assumed that could be used without the rest of stdio. OK. Never mind. :)
You won't get all of stdio but you will get a piece of it - the part that does the actual string formatting (%s %d etc) but none of the terminal related code which is another chunk.

However if you are using the sprintf family you should control which printf converters are actually used, otherwise you will be getting the default which supports a bunch of them "%d %u %x %X %o %n %i %p %s %c %ld %lu %lx %lX %lo %ln %li %lp" IIRC. You can control it with a '#pragma printf = "%u ...' type of pragma (an example with %u and more here, and you can have an empty string there too if you don't need any) and similarly if you use the sscanf family you can control the % converters included with a "#pragma scanf =...". Keep in mind if you use these pragmas, you will be getting the printf/scanf core included into your binary but that should be ok since you wouldn't want to use them unless you were calling sprintf or sscanf already.
https://www.z88dk.org/wiki/doku.php?id= ... e_easy_way

I was a little confused because your error message is about "fflush()" being in the compile and I don't recall how that would get pulled in for a sprintf/sscanf. sprintf/sscanf sets up a fake FILE structure to print/scan to/from and the library has a build-time option to verify that FILE structures are valid before attempting to use them. This might cause a problem if you're attempting to use "#pragma output CLIB_FOPEN_MAX = -1" because then the library wouldn't have a list of valid FILEs to verify against. However I can see the default build of the z80 library (newlib here which you are using) does not verify the FILE structure at all ( https://github.com/z88dk/z88dk/blob/mas ... ib.m4#L233 ) so I'm not sure how fflush() is coming into it or why you can't use that pragma. These things are small but I'll try to have a look later.

In terms of the exx set, stdio (including sprintf and sscanf) will use the exx set because the code keeps some counters and independent parameters in the exx set while the main set is used to generate output (and some converters like the 32-bit integer and 64-bit integer will also use the exx set independent of this).
Post Reply