Problems with using esxdos.h

[Stefan123]

I'm experimenting with the new esxdos support in z88dk but I can't get it to work.

I have the following simple test program, which tries to open the file test.txt located in the directory D:\Spectrum\test\:

Code: Select all

int main(void)
{
    unsigned char handle;

    errno = 0;
    esxdos_ram_m_setdrv(ESXDOS_DRIVE_CURRENT);
    if (errno)
    {
        zx_border(INK_GREEN);
        z80_delay_ms(10000);
        return 1;
    }

    errno = 0;
    handle = esxdos_ram_f_open("test.txt", ESXDOS_MODE_R);
    if (errno)
    {
        zx_border(INK_RED);
        z80_delay_ms(10000);
        return 1;
    }

    ...
}

I run my program in ZEsarUX using the following options:

Code: Select all

zesarux.exe --machine tbblue --enable-mmc --enable-divmmc-ports --mmc-file extras\media\disk_images\tbblue.mmc --enable-esxdos-handler --esxdos-root-dir D:\Spectrum\test --quickexit --tape D:\Spectrum\test\filetest.tap

However, the call to esxdos_ram_f_open() fails with errno being set. What am I doing wrong?

I haven't found any examples of using esxdos.h in z88dk. Alvin, do you have any working examples?

[Stefan123]

I forgot to mention that errno is set to ESXDOS_ENOENT (no such file) after the call to esxdos_ram_f_open().

[alvin]

I haven't tried disk tests yet but I will try tonight. I have been doing dot commands the last couple of days.

zcc +zxn -vn -startup=30 -subtype=dot -clib=sdcc_iy -SO3 --max-allocs-per-node200000 test.c -o test -create-app

will generate a dot command using the new rom driver (startup=30) which uses rst 16 to print so that saves on memory. The command line is set up by default (use argc, argv on main) and the return value is an esxdos error with 0 = no error and anything else producing an error in basic.

For the disk commands I'm a little worried the emulator authors aren't emulating properly yet because they are working from much less information than I have (see https://github.com/z88dk/z88dk/blob/mas ... _esxdos.m4 and the implementation comments in https://github.com/z88dk/z88dk/tree/mas ... xdos/z80). The one thing in particular is that esxdos requires all file functions to provide a drive to act on but it's possible to use the "current drive" by using "*" as drive number. That's what z88dk is doing to reduce the number of parameters passed to these functions. I'm not sure the emulators handle this but I will find this out tonight.

[alvin]

I made a change after the last nightly build to fix the error codes returned from the esxdos functions.

This is my test program which is Jim Bagley's BMPtoNext.c for the PC, now compiled for the actual Next:

Code: Select all

// zcc +zxn -vn -subtype=dot -startup=30 -clib=sdcc_iy -SO3 --max-allocs-per-node200000 --opt-code-size bmp2next.c -o bmp2next -lm -create-app
// (bug report pending) zcc +zxn -vn -subtype=dot -startup=30 -clib=new bmp2next.c -o bmp2next -lm -create-app 

#pragma printf "%u %s"
#pragma output CLIB_EXIT_STACK_SIZE = 3

#include <stdio.h>
#include <math.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <stdarg.h>
#include <errno.h>
#include <input.h>
#include <arch/zxn/esxdos.h>

uint16_t header[0x36/2];
uint8_t  palindex[256];

// anonymous unions are C11 :(

union _umem
{
   uint8_t  palette[0x400];
        uint8_t  buffer[0x400];
};
union _umem umem;

float_t r,g,b;
uint8_t ri,gi,bi;

unsigned char fin;
unsigned char fout;

// cleanup on exit

void cleanup_fin(void)
{
        esxdos_f_close(fin);
}

void cleanup_fout(void)
{
        esxdos_f_close(fout);
}

void cleanup_lf(void)
{
        printf("\n");
}

// generate custom esxdos error report

int error(char *fmt, ...)
{
        static unsigned char ebuf[65];
        
   va_list v;
   va_start(v, fmt);
   
   vsnprintf(ebuf, sizeof(ebuf), fmt, v);
   ebuf[strlen(ebuf)-1] += 0x80;
   
   return (int)ebuf;
}

uint16_t size;
uint32_t remaining, total;

int main(int argc, char **argv)
{
        uint16_t i;
        
        // help information
        
   if (argc < 2)
        {
                printf("BMP2Next Usage :-\nBMP2Next src.bmp [dst.bmp]\n");
      return 0;
        }
        
        // open input file
        
        fin = esxdos_f_open(argv[1],(argc == 2) ? ESXDOS_MODE_R|ESXDOS_MODE_W|ESXDOS_MODE_OE : ESXDOS_MODE_R|ESXDOS_MODE_OE);
        if (errno)
                return error("%u: Can't open %s",errno,argv[1]);

        atexit(cleanup_fin);

        // read header from bitmap file
        
        esxdos_f_read(fin,header,0x36*1);
        esxdos_f_read(fin,umem.palette,256*4);
        if (errno)
                return error("%u: Can't read header from %s",errno,argv[1]);
        
        // compute 

        for (i = 0; i < 256; ++i)
        {
                r  = umem.palette[i*4+2];    // bmp palette is ordered B/G/R/A * 256 after the 0x36 byte header
                r += (255.0f/7.0f)/2.0f;     // add half a unit to round up colours to closest unit
                ri = r/(255.0f/7.0f);        // get an int from 0 to 7 for Red
                
                g  = umem.palette[i*4+1];
                g += (255.0f/7.0f)/2.0f;
                gi = g/(255.0f/7.0f);        // get an int from 0 to 7 for Green
                
                b  = umem.palette[i*4+0];
                b += (255.0f/3.0f)/2.0f;
                bi = b/(255.0f/3.0f);        // get an int from 0 to 3 for Blue
                
      // palette index for this pixel value in the bmp is the RRRGGGBB index for Next
                palindex[i] = ((ri&7)<<5) + ((gi&7)<<2) + (bi&3);
                
                // in-place modification of palette entries
                
                umem.palette[i*4+3] = 0;
                umem.palette[i*4+2] = (uint8_t)(((i>>5)&7)*(255.0f/7.0f));
                umem.palette[i*4+1] = (uint8_t)(((i>>2)&7)*(255.0f/7.0f));
                umem.palette[i*4+0] = (uint8_t)(((i>>0)&3)*(255.0f/3.0f));
        }
        
        // open output file
        
        if (argc == 2)
        {
                // if only one filename overwrite the original

                esxdos_f_seek(fin,0L,ESXDOS_SEEK_SET);
                if (errno)
                   return error("%u: Can't seek on input file",errno);
                
                fout = fin;
        }
        else
        {
                fout = esxdos_f_open(argv[2],ESXDOS_MODE_W|ESXDOS_MODE_CT);
                if (errno)
                   return error("%u: Can't open %s for writing",errno,argv[2]);

                atexit(cleanup_fout);
        }
        
        // write bitmap header to output file
        
        esxdos_f_write(fout,header,0x36*1);
        esxdos_f_write(fout,umem.palette,256*4);
        if (errno)
                return error("%u: Can't write header to %s",errno,(fin == fout) ? argv[1] : argv[2]);

        // process bitmap 1k pixels at a time
        
        printf("\nProgress:  0%%\x08\x08\x08");
        atexit(cleanup_lf);
        
        total = remaining = header[9] * (uint32_t)header[11];
        while (remaining && (size = esxdos_f_read(fin,umem.buffer,(remaining > sizeof(umem.buffer)) ? sizeof(umem.buffer) : remaining)))
        {
                remaining -= size;
                
                if (fin == fout)
                {
                        esxdos_f_seek(fin,size,ESXDOS_SEEK_BWD);
                        if (errno)
                                return error("%u: Can't seek on output file",errno);
                }
                
                for (i = 0; i != size; ++i)                      // go through each pixel
                        umem.buffer[i] = palindex[umem.buffer[i]];    // and change it to its new index from the Next palette.
                
                esxdos_f_write(fout,umem.buffer,size);
                if (errno)
                        return error("%u: Error writing to output file",errno);
                
                printf("%02u%%\x08\x08\x08", ((total-remaining)*100)/total);
                
                // allow user to interrupt
                
                if (in_key_pressed(IN_KEY_SCANCODE_SPACE | 0x8000))
                {
                        in_wait_nokey();
                        error("L Break into Program");
                }
        }
        
        if (remaining)
                error("%u: Premature end to reading input file",errno);
        
        printf("done!");
        
        return 0;
}

In ZEsurUX with esxdos handler I am getting past the open of the input file and reads of the header if the input file and output file are distinct. It is failing with error code at the open of the output file. So I think there are some issues with open mode flags with the esxdoshandler in ZEsurUX and I am trying to figure out how to make an mmc image to test using the actual esxdos code instead of through the handler to find out if the problem is mine or the emulation's.

[alvin]

I have working disk access under emulation using the esxdos handler in ZEsurUX. The mode bits that have been defined are not quite correct and have to be fixed up.

Here's a working dot command that converts 256-colour bmp images of any size to 256-colour bitmap images using the Next's RRRGGGBB colour scheme:
https://drive.google.com/.../0B6XhJJ33x ... ../view...

Comments:

The file "zxnext.bmp" is the original and the file output by the Next is "zx-new.bmp". The program was run under ZEsurUX emulation with the esxdos handler on (allows access to files on PC disk) using:

.bmp2next zxnext.bmp zx-new.bmp

I had to change mode flags on the file opens to work on ZEsurUX.

The open for the input file:

fin = esxdos_f_open(argv[1],(argc == 2) ? ESXDOS_MODE_R|ESXDOS_MODE_W|ESXDOS_MODE_OE : ESXDOS_MODE_R|ESXDOS_MODE_OE);

The path for argc==2 has no output file specified and opens the file for RW. This does not work on ZEsurUX.

The other path opens the file for reading and is ok.

The open for the output file:

fout = esxdos_f_open(argv[2],ESXDOS_MODE_W|ESXDOS_MODE_OC); //|ESXDOS_MODE_CT);

ESXDOS_MODE_OC (open create) works on ZEsurUX but it's actually an append mode write. So I have some wrong documentation from Miguel several years ago. The mode bits will have to be fixed.

Total emulated running time was 25 seconds at 3.5 MHz but esxdos disk access is likely not emulated timing accurate. This is for a 449078 byte file 1120x400.

[alvin]

Sorry bad link: https://drive.google.com/file/d/0B6XhJJ ... sp=sharing

[Stefan123]

I downloaded the latest z88dk Windows snapshot from 2017-08-03 but I still can't get esxdos calls to work.

I start ZEsarUX in the following way:

zesarux.exe --machine tbblue --enable-mmc --enable-divmmc-ports --mmc-file extras\media\disk_images\tbblue.mmc --enable-esxdos-handler --esxdos-root-dir D:\Spectrum\ZEsarUX_win-5.1\extras\media\spectrum\esxdos_handler\0.8.6_TBBLUE

I must confess that I'm a bit confused by all the MMC options and the ZEsarUX GUI in general. However, if I start ZEsarUX in this way I can type .ls in the BASIC prompt and see the contents of the D:\Spectrum\ZEsarUX_win-5.1\extras\media\spectrum\esxdos_handler\0.8.6_TBBLUE directory where I have also created a test file called test.txt.

Is this how you start ZEsarUX? Should some of the MMC options be removed when using the esxdos-handler?

My test program is the following:

Code: Select all

int main(void)
{
    unsigned char handle;

    errno = 0;
    esxdos_ram_m_setdrv(ESXDOS_DRIVE_CURRENT);
    if (errno)
    {
        printf("Errno in esxdos_ram_m_setdrv(): %d\n", errno);
        return 1;
    }

    errno = 0;
    handle = esxdos_ram_f_open("test.txt", ESXDOS_MODE_R | ESXDOS_MODE_OE);
    if (errno)
    {
        printf("Errno in esxdos_ram_f_open(): %d\n", errno);
        return 1;
    }

    ...
}

I compile this test program with the following command line:

Code: Select all

zcc +zx -vn -O3 -startup=30 -clib=new test.c -o test -create-app

This program fails with errno ESXDOS_ENOENT (no such file) after the call to esxdos_ram_f_open("test.txt", ESXDOS_MODE_R | ESXDOS_MODE_OE). I use the same flags in esxdos_ram_f_open() as you do in the bmp2next.c program. I can see the test.txt file when doing .ls in the BASIC prompt. Am I doing something wrong or have I missed something? Have you only tested using esxdos from a dot command or have you also used the ram variants of the esxdos calls from a program?

I see that you don't call esxdos_m_setdrv() or esxdos_m_getdrv() in bmp2next.c. When are those calls needed? I have tried to skip them but I still get the same error.

I have also tried to run my test program in CSpect but then it fails with errno ESXDOS_ENODRV (no such drive) after the call to esxdos_ram_f_open("test.txt", ESXDOS_MODE_R | ESXDOS_MODE_OE).

[Stefan123]

Yay, I finally got it to work! While grasping at straws, I studied your bmp2next.c file more carefully and noticed that you compile with the SDCC compiler and had written a comment about a bug when using the SCCZ80 compiler with esxdos. So when I tried the following command line to compile my test program it works:

Code: Select all

zcc +zx -vn -SO3 -startup=30 -clib=sdcc_iy --max-allocs-per-node200000 test.c -o test -create-app

I can now open and read a file :-) I don't call esxdos_m_setdrv() or esxdos_m_getdrv() since they don't seem to be needed.

My test program still fails in CSpect when trying to open a file (ESXDOS_ENODRV) but I guess that the esxdos support in CSpect is less complete than in ZEsarUX.

[Stefan123]

By the way, I think esxdos.h should include stddef.h for its use of size_t.

[alvin]

Yay, I finally got it to work! While grasping at straws, I studied your bmp2next.c file more carefully and noticed that you compile with the SDCC compiler and had written a comment about a bug when using the SCCZ80 compiler with esxdos. So when I tried the following command line to compile my test program it works:

The bug is connected to something else but I may have messed up the c interface for sccz80. I will doublecheck them. It's sometimes hard to keep things straight when you have to write a lot of functions, with some of them taking parameters left to right and others right to left.

By the way, I think esxdos.h should include stddef.h for its use of size_t.

Yep I will stick it in.

My test program still fails in CSpect when trying to open a file (ESXDOS_ENODRV) but I guess that the esxdos support in CSpect is less complete than in ZEsarUX.

In Mike's examples he's always getting the drive number and then setting it before every call. So I don't think he knows that '*' is a special drive number that means the current drive. That's being used in z88dk to reduce the number of parameters passed to the esxdos functions.

I don't call esxdos_m_setdrv() or esxdos_m_getdrv() since they don't seem to be needed.

Yeah you don't need to set it. esxdos can do several partitions and each one is a drive. So you can change between partitions by calling setdrv with a suitable drive number. If you don't set it, you just continue using the drive where your program was started from.

[Stefan123]

Yeah, I understand that it is a lot of work to maintain different versions of newlib for different calling conventions, C language level support etc. I assume there are historical reasons for having different calling conventions in the two C compilers. Maybe it would pay off to invest in unifying the calling conventions between the compilers so there would only be one flavor of newlib?

I worked with embedded systems and realtime operating systems a long time ago and I remember the hassle with different callings conventions for varargs, supporting both hardware and software floating point, big endian vs little endian etc.

By the way, there is a question I have been thinking of asking for some time. Is there any relation whatsoever between z88dk's newlib and the Red Hat-maintained open source project Newlib (https://sourceware.org/newlib/) or do they just happen to have the same name?

[alvin]

sccz80 compiles should work now. I've also confirmed that the mode flags we've defined are correct and the behaviour in the emulators is incorrect for a number of things. Hopefully the emulators will catch up soon.

Maybe it would pay off to invest in unifying the calling conventions between the compilers so there would only be one flavor of newlib?

An option has been added to make sccz80 use more normal right to left parameter passing but I've not yet invested the time to fix things in that direction yet. There are a number of other issues to make it possible to mix object files between sccz80 and zsdcc and it's best to do all of that at once since it's a fairly big job. Even with the same parameter order, the calling is still not quite the same as zsdcc pushes single bytes for char and sccz80 can use the same entry for fastcall and non-fastcall entry points.

Is there any relation whatsoever between z88dk's newlib and the Red Hat-maintained open source project Newlib (https://sourceware.org/newlib/) or do they just happen to have the same name?

Just a coincidence.

[Stefan123]

I downloaded the latest z88dk Windows snapshot from 2017-08-05 but I still get errno ESXDOS_ENOENT when calling esxdos_ram_f_open() if compiling with the SCCZ80 compiler. When using the SDCC compiler it works fine.

[alvin]

I can't see anything wrong with it... I'll try some test programs when I get home later tonight.

[alvin]

It's a bug in ZEsarUX.

Cesar has mentioned that he's not yet emulating a distinction between accessing esxdos from within esxdos and from without. From within esxdos, some parameters are passed in HL. But when accessed from ram (ie when esxdos is not paged in), the same parameters must be passed in IX. ZEsarUX is always expecting the parameter in HL.

In zsdcc compiles, by accident, IX and HL will contain the same value when calling esxdos from ram so zsdcc compiles will always work. With sccz80, otoh, the parameter is correctly in IX and HL is something else so the call fails.

This will work for sccz80 under emulation:

Code: Select all

#define __ESXDOS_DOT_COMMAND

#include <stdio.h>
#include <arch/zx.h>
#include <arch/zx/esxdos.h>
#include <errno.h>
#include <input.h>

int main(void)
{
    unsigned char handle;


    errno = 0;
    esxdos_m_setdrv(ESXDOS_DRIVE_CURRENT);
    if (errno)
    {
        printf("Errno in esxdos_ram_m_setdrv(): %d\n", errno);
        return 1;
    }

    errno = 0;
    handle = esxdos_f_open("test.txt", ESXDOS_MODE_R | ESXDOS_MODE_OE);
    if (errno)
    {
        printf("Errno in esxdos_ram_f_open(): %d\n", errno);
                  in_wait_nokey();
        return 1;
    }

         in_wait_nokey();
         return 0;
}

The define at the top will get esxdos.h to select the dot version of the esxdos calls where params are in hl (without it, the ram versions are called). So this hack will allow code to work with ZEsarUX as it is now.

We just have to wait until this is fixed up by Cesar.

[Stefan123]

Thanks for taking the time and finding the root cause. I will use your workaround until Cesar updates ZEsarUX.
It's so convenient being able to load files (particularly layer 2 screen files) using esxdos

[alvin]

It's so convenient being able to load files (particularly layer 2 screen files) using esxdos

Yes it is! It makes life much simpler.