Issues when migrating back to Classic from newlib

Requests for features
User avatar
jorgegv
Well known member
Posts: 287
Joined: Wed Nov 18, 2020 5:08 pm

Re: Issues when migrating back to Classic from newlib

Post by jorgegv »

OK, so here is the new PR: https://github.com/z88dk/z88dk/pull/2006 (I have withdrawn the old one).

Much cleaner, only contains my commits, and it's much, much easier to review :-)

Please share your feedback, Dom ;-)
clebin
Member
Posts: 15
Joined: Wed Mar 23, 2022 6:28 pm

Re: Issues when migrating back to Classic from newlib

Post by clebin »

I'm getting the SP1 u_malloc errors on sccz80 - not sure if this was fixed. How do I prevent these?

Code: Select all

/home/build/z88dk/build/z88dk/libsrc/_DEVELOPMENT/temp/sp1/zx/sprites/asm_sp1_AddColSpr.asm:40: error: undefined symbol: _u_malloc
  ^---- _u_malloc
/home/build/z88dk/build/z88dk/libsrc/_DEVELOPMENT/temp/sp1/zx/sprites/asm_sp1_AddColSpr.asm:200: error: undefined symbol: _u_free
  ^---- _u_free
/home/build/z88dk/build/z88dk/libsrc/_DEVELOPMENT/temp/sp1/zx/sprites/asm_sp1_CreateSpr.asm:47: error: undefined symbol: _u_malloc
  ^---- _u_malloc
/home/build/z88dk/build/z88dk/libsrc/_DEVELOPMENT/temp/sp1/zx/sprites/asm_sp1_CreateSpr.asm:65: error: undefined symbol: _u_malloc
  ^---- _u_malloc
/home/build/z88dk/build/z88dk/libsrc/_DEVELOPMENT/temp/sp1/zx/sprites/asm_sp1_CreateSpr.asm:221: error: undefined symbol: _u_free
  ^---- _u_free
User avatar
jorgegv
Well known member
Posts: 287
Joined: Wed Nov 18, 2020 5:08 pm

Re: Issues when migrating back to Classic from newlib

Post by jorgegv »

You can just define the needed functions like in libsrc/sprites/software/sp1/zx/examples/ex0.c

The examples in that directory are a real gold mine for SP1 usage.
clebin
Member
Posts: 15
Joined: Wed Mar 23, 2022 6:28 pm

Re: Issues when migrating back to Classic from newlib

Post by clebin »

Thanks again jorgegv!

That sort of worked - except it's now doing something very weird that never happened with newlib. Whenever I press a key, any key, my sprite disappears. Even if I put it in a loop with nothing else changing, like this:

Code: Select all

        while (1)
        {
            // draw cursor
            sp1_MoveSprAbs(
                cursorSp,
                &fullscreen,
                0,
                8,
                8,
                0, 0);
            sp1_UpdateNow();
        }
Is this something to do with the heap location or size? I'm initialising it as per the example:

Code: Select all

void *u_malloc(uint size)
{
    return malloc(size);
}

void u_free(void *addr)
{
    free(addr);
}

int main()
{
    // Initialize heap
    heap = 0L;                  // heap is empty
    sbrk((void *)40000, 10000); // add 40000-49999 to malloc
What could be happening?
User avatar
jorgegv
Well known member
Posts: 287
Joined: Wed Nov 18, 2020 5:08 pm

Re: Issues when migrating back to Classic from newlib

Post by jorgegv »

Can you post your complete code and compilation line? You need to define the Sprite somewhere...
clebin
Member
Posts: 15
Joined: Wed Mar 23, 2022 6:28 pm

Re: Issues when migrating back to Classic from newlib

Post by clebin »

The program is very big now, but I'll try and share the pertinent bits (I can also share the private git repo with you?) The sprite code works ok in newlib and is mostly copied from my previous game, so I'm stumped. =(

Initialise sp1:

Code: Select all

    sp1_Initialize(
        SP1_IFLAG_MAKE_ROTTBL |
            SP1_IFLAG_OVERWRITE_TILES |
            SP1_IFLAG_OVERWRITE_DFILE,
        INK_WHITE |
            PAPER_BLACK |
            BRIGHT,
        ' ');
        
Create the sprite:

Code: Select all

void initialiseCursor()
{
    cursorSp = getNewMaskedSprite((int *)cursorSprite, 2, 2);
}

struct sp1_ss *getNewMaskedSprite(uint8_t *sprite, uint8_t spCols, uint8_t spRows)
{
    struct sp1_ss *sp;

    sp = sp1_CreateSpr(SP1_DRAW_OR2LB, SP1_TYPE_2BYTE, spRows + 1, (int)sprite, 0);

    for (int i = 1; i < spCols; i++)
    {
        sp1_AddColSpr(sp, SP1_DRAW_OR2, SP1_TYPE_2BYTE, (int)&sprite[(i * spRows * 16) + (i * 16)], 0);
    }
    sp1_AddColSpr(sp, SP1_DRAW_OR2RB, SP1_TYPE_2BYTE, 0, 0);

    return sp;
}
Compile:

Code: Select all

compile:
	zcc +zx \
	-v \
	-lm \
	-Wall \
	-lndos \
	-lsp1 \
	-m \
	-compiler sccz80 \
	@zproject.lst \
	-o o/spec-wars.bin \
	-pragma-include:zpragma.inc
Pragmas:

Code: Select all

#pragma output CRT_ORG_CODE = 0x5DCF
#pragma output CRT_ORG_DATA = 0
#pragma output CRT_ORG_BSS = 0
#pragma output REGISTER_SP = 0xBFFF
User avatar
jorgegv
Well known member
Posts: 287
Joined: Wed Nov 18, 2020 5:08 pm

Re: Issues when migrating back to Classic from newlib

Post by jorgegv »

In your example code, you are setting the area between 40000 and 49999 as the heap. But them in your pragma file you initially set the stack to point at 0xBFFF (49151 in decimal), which is completely inside your heap area. Depending on your heap use, this is a recipe for disaster :-)

I suggest you design a complete memory map for your game (if you still haven't), taking into account _all_ the areas that you need to track (code, data, heap, ISR and banking area), and configure the pragmas accordingly.

In my code I usually define the heap manually: I setup the pragmas so that the library does not automatically define it, then I declare a byte array of the desired size, and then I configure the heap to use that byte array as the memory block.
Timmy
Well known member
Posts: 392
Joined: Sat Mar 10, 2012 4:18 pm

Re: Issues when migrating back to Classic from newlib

Post by Timmy »

There is also a larger sp1 example in: z88dk/libsrc/_DEVELOPMENT/EXAMPLES/zx/demo_sp1/demo2/ (It's a bit old so I don't know if it still works.)

Another tip I usually do in my games is to not ever free your sprites but reusing them by changing the pointer to the sprite data. This will reduce memory allocations.

But I use an emulator that shows me the memory used so I know how to set up my memory map.
clebin
Member
Posts: 15
Joined: Wed Mar 23, 2022 6:28 pm

Re: Issues when migrating back to Classic from newlib

Post by clebin »

Thanks both. That was stupid of me not reducing the heap size there. The thing is, I should be using next to no heap. I also like to reuse and recycle sprites etc as much as possible, creating arrays of the maximum size of anything I'm going to need, so I'm not using malloc anywhere in my own code. In this case, I've literally only got the one 16x16 sprite which is my cursor. I'm using a reasonable amount of stack to be fair, as I'm doing some recursive A* type pathfinding, but this happens right at the beginning of the program and before the pathfinding function is even called, so the stack should be tiny too.

I've played with the heap settings, putting it well after code+BSS and well clear of the stack and I still get the magic disappearing sprite. I'll try your tip jorgegv and create an ASM array org'ed at the location of my heap. How does the CLIB_MALLOC_HEAP_SIZE pragma setting work with/alongside the sbrk() function?

As an experiment I tried putting the heap between 0xC000 and the start of sp1 at 0xD000, figuring that I'll not need it while another bank is switched in, but that didn't work. I'm quite disheartened right now as I felt I'd cleared the other technical roadblocks to making the game I wanted to make.
Timmy
Well known member
Posts: 392
Joined: Sat Mar 10, 2012 4:18 pm

Re: Issues when migrating back to Classic from newlib

Post by Timmy »

Just a quick note:

stack is not heap. they are two different things. the stack is needed for z88dk, and the heap is used in sp1.

in sp1, every time you create a sprite, a quite big amount of memory is allocated in the heap. it's necessary for sp1 to function. it's a fixed size depending on how large your sprite is.

i'd suggest you not to use malloc yourself and let it only be used by sp1.

sp1 uses a lot of memory, but the result is that you don't need to worry about sprites and it's probably the best and fastest generic (masked) sprite library for the spectrum.

if you don't need to use sprites then it's so much easier to just skip sp1, but then you'll have to write in assembly (or perhaps use putsprite() or some other routine) (i never used putsprite myself so i have no opinion about it.)
User avatar
jorgegv
Well known member
Posts: 287
Joined: Wed Nov 18, 2020 5:08 pm

Re: Issues when migrating back to Classic from newlib

Post by jorgegv »

Hey,

I started migrating RAGE1 back to Classic and it still is a work in progress (it's been stalled for some months :-) ) but when I started doing it (I started this forum thread at that time) I already migrated the memory management back to Classic. I'm pasting my snippet for memory initialization here (you can ignore the non-relevant bitse.g. the 128-mode defines and focus on the 48K mode):

Code: Select all

// memory init depends on the target

// in 128K mode, heap is at the top of the 0x5B00-0x7FFF area
#ifdef BUILD_FEATURE_ZX_TARGET_128
    #define MALLOC_HEAP_START       ((unsigned char *)(0x8000 - MALLOC_HEAP_SIZE))
#endif

// in 48K mode, we define a heap in the BSS segment
#ifdef BUILD_FEATURE_ZX_TARGET_48
    #define MALLOC_HEAP_START       (&_rage1_heap[0])
    unsigned char _rage1_heap[ MALLOC_HEAP_SIZE ];
#endif

// memory initialization
//unsigned char *_malloc_heap;
long heap;
void init_memory(void) {
//    _malloc_heap = MALLOC_HEAP_START;
//    heap_init( MALLOC_HEAP_START, MALLOC_HEAP_SIZE );
    mallinit();
    sbrk( MALLOC_HEAP_START, MALLOC_HEAP_SIZE );

#ifdef BUILD_FEATURE_ZX_TARGET_128
    // initial memory bank
    memory_current_memory_bank = 0;
#endif
}
The commented out code is the (currently working) code for newlib+SDCC. The new one is for SDCC+Classic.

Hope this helps a bit more.
clebin
Member
Posts: 15
Joined: Wed Mar 23, 2022 6:28 pm

Re: Issues when migrating back to Classic from newlib

Post by clebin »

This is so weird. Here's a (virtually) empty project with the same sprite disappearing magic trick. Note: I'm compiling on a M1 Mac in case there's some platform specific weirdness.

https://github.com/clebin/spritepocalypse
Timmy
Well known member
Posts: 392
Joined: Sat Mar 10, 2012 4:18 pm

Re: Issues when migrating back to Classic from newlib

Post by Timmy »

Took me a long while to figure out what was wrong. The problem with your code is that you need to set up interrupts for it to work. yes, sp1 also requires interrupts.

A very quick and dirty solution for your current code is to put "intrinsic_di();" immediately after main(), and include "#include <intrinsic.h>". (I actually didn't used that, I used "#asm di #endasm" instead.)

For a proper set up of an im2 routine (which doesn't need to do anything other than just EI; RETI), see the examples.
clebin
Member
Posts: 15
Joined: Wed Mar 23, 2022 6:28 pm

Re: Issues when migrating back to Classic from newlib

Post by clebin »

Woohoo, you star! Thanks Timmy and jorgegv. As I'm using 128k banks I had to move the table further down in memory with zx_im2_init() but I have my cursor sprite at last! It's shown me I've still a lot to learn but one step (and game) at a time...
User avatar
jorgegv
Well known member
Posts: 287
Joined: Wed Nov 18, 2020 5:08 pm

Re: Issues when migrating back to Classic from newlib

Post by jorgegv »

Mmm I don't think SP1 requires interrupts. SP1 internally uses IY, and the default ROM interrupt service routine for Mode 1 assumes IY has a fixed constant value. So the moment you get an interrupt inside a SP1 routine (and corrupt IY), you're screwed.

And that's the reason it works if you disable interrupts, of course.
Last edited by jorgegv on Sun Nov 20, 2022 9:04 am, edited 1 time in total.
User avatar
jorgegv
Well known member
Posts: 287
Joined: Wed Nov 18, 2020 5:08 pm

Re: Issues when migrating back to Classic from newlib

Post by jorgegv »

clebin wrote: Sun Nov 20, 2022 9:02 am Woohoo, you star! Thanks Timmy and jorgegv. As I'm using 128k banks I had to move the table further down in memory with zx_im2_init() but I have my cursor sprite at last! It's shown me I've still a lot to learn but one step (and game) at a time...
...and the reason is that you have setup an IM2 routine, which makes everything work with IY :-)
Timmy
Well known member
Posts: 392
Joined: Sat Mar 10, 2012 4:18 pm

Re: Issues when migrating back to Classic from newlib

Post by Timmy »

You are definitely right that sp1 doesn't need to use an interrupt. But it's almost always useful to set up interrupts _now_ until later, when your memory usage is almost full and then you find out you need an additional 257 byte table. :)

And congratulations for getting it working! Just to make sure, on 128K and higher models, you should place both the interrupt table and the ISR between 32768 and (including) 49151.

(I haven't touched sp1 in years now, so I've spent lot of time to just make your code work first -- while not having the php part, and so many different files -- and then I had to compare your code against a working one to see what you were missing. So that took me some hours on a saturday evening. I will try not to do that again, but I probably can't resist doing stuff on saturday evenings. :) )

There is a good reason the demo2 code (wow, it's from 2012, 10 years ago!) is there, because it was the results of many months of knowledge of me learning sp1. And it gives a good boilerplate code to work from, with almost everything except for sound/music.
clebin
Member
Posts: 15
Joined: Wed Mar 23, 2022 6:28 pm

Re: Issues when migrating back to Classic from newlib

Post by clebin »

Timmy wrote: Sun Nov 20, 2022 11:38 am (I haven't touched sp1 in years now, so I've spent lot of time to just make your code work first -- while not having the php part, and so many different files -- and then I had to compare your code against a working one to see what you were missing. So that took me some hours on a saturday evening. I will try not to do that again, but I probably can't resist doing stuff on saturday evenings. :) )
Wow, that is going beyond the call of duty. If I ever run into you, I definitely owe you some beers!
clebin
Member
Posts: 15
Joined: Wed Mar 23, 2022 6:28 pm

Re: Issues when migrating back to Classic from newlib

Post by clebin »

Me again, with a semi-related question...

The following bit of code works fine now, but the zx_im2_init() prevents me from being able to use the FRAMES counter.

With printf("%d", FRAMES) the number output is always 24 in my case. Without that line, FRAMES counts up as expected. What do I need to to re-enable FRAMES after that call?

Code: Select all

intrinsic_di();
zx_im2_init(0xBC00, 0xBD);
intrinsic_ei(); // enable interrupts
User avatar
jorgegv
Well known member
Posts: 287
Joined: Wed Nov 18, 2020 5:08 pm

Re: Issues when migrating back to Classic from newlib

Post by jorgegv »

clebin wrote: Sun Dec 18, 2022 12:27 pm Me again, with a semi-related question...

The following bit of code works fine now, but the zx_im2_init() prevents me from being able to use the FRAMES counter.

With printf("%d", FRAMES) the number output is always 24 in my case. Without that line, FRAMES counts up as expected. What do I need to to re-enable FRAMES after that call?

Code: Select all

intrinsic_di();
zx_im2_init(0xBC00, 0xBD);
intrinsic_ei(); // enable interrupts
The FRAMES sysvar is maintained by the ROM ISR routine, but when you activate IM2 that routine is not called, but your ISR is instead.

If you want to keep FRAMES Up to date your ISR should call the ROM ISR.

Or update the FRAMES variable (or an equivalente one) yourself in your ISR, it's just a simple counter...
User avatar
jorgegv
Well known member
Posts: 287
Joined: Wed Nov 18, 2020 5:08 pm

Re: Issues when migrating back to Classic from newlib

Post by jorgegv »

Also remember to call ROM ISR with IY = $5C3A, ROM expects that value in IY always
clebin
Member
Posts: 15
Joined: Wed Mar 23, 2022 6:28 pm

Re: Issues when migrating back to Classic from newlib

Post by clebin »

jorgegv wrote: Sun Dec 18, 2022 3:05 pm Also remember to call ROM ISR with IY = $5C3A, ROM expects that value in IY always
Thanks jorgegv, that was enough to get me started. I chose to increment it myself:

Code: Select all

void framecounter()
{
    if (FRAMES + 1 > FRAMES) // check overflow
        FRAMES++;
    else
        FRAMES = 0;
}

int main()
{
    intrinsic_di();
    zx_im2_init(0xBC00, 0xBD);
    add_raster_int(framecounter);
    intrinsic_ei();
Does that look acceptable?
User avatar
jorgegv
Well known member
Posts: 287
Joined: Wed Nov 18, 2020 5:08 pm

Re: Issues when migrating back to Classic from newlib

Post by jorgegv »

I would just define a new framecount variable and use it instead of FRAMES:

Code: Select all

uint8_t top_frames = 0;
uint16_t frames = 0;

void framecounter() {
    if ( ! ++frames )
        top_frames++;
}
...
You dont need to check for overflow, the CPU handles It by itself.

FRAMES sysvar is 24 bits, that's the reason for the strange update code.
User avatar
dom
Well known member
Posts: 2072
Joined: Sun Jul 15, 2007 10:01 pm

Re: Issues when migrating back to Classic from newlib

Post by dom »

There's a precanned tick count ISR in the library already:

Code: Select all

add_raster_int(tick_count_isr);
Which mutates the counter long tick_count;
clebin
Member
Posts: 15
Joined: Wed Mar 23, 2022 6:28 pm

Re: Issues when migrating back to Classic from newlib

Post by clebin »

dom wrote: Sun Dec 18, 2022 9:02 pm There's a precanned tick count ISR in the library already:

Code: Select all

add_raster_int(tick_count_isr);
Which mutates the counter long tick_count;
Thanks both! I went with tick_count_isr.
Post Reply