[z88dk-dev] z80asm bug: defc not assigning correct value

Post by **alvin** » Fri Dec 04, 2015 3:11 am

test.asm:

SECTION AAA
org 0
SECTION BBB
SECTION CCC

SECTION AAA
defw __i_fcntl_fdstruct_1

SECTION BBB

__i_fcntl_fdstruct_1:
defb 205

SECTION CCC
defw __i_fcntl_fdstruct_1

defc __i_fcntl_fdstruct_2 = __i_fcntl_fdstruct_1

z80asm -b -o=test test.asm

test.map shows this:

__i_fcntl_fdstruct_1 = 0002, L: test
__i_fcntl_fdstruct_2 = 0003, L: test

The two labels are not equal despite the defc.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/ ... 1&iu=/4140

pscust · Post by **pscust** » Sun Dec 06, 2015 11:29 am

Alvin,

Thanks for the report.

The cause was that the DEFC expression was being computed in section CCC,
but with terms refering to section BBB. The expression was computed at the
compile phase with a wrong base address.

The problem should now be fixed. Any DEFC expression that depends on
symbols from other modules or sections is now postponed to the link phase.

Please let me know of further issues.

Regards,
Paulo

On Fri, Dec 4, 2015 at 3:11 AM, alvin (alvin_albrecht@...) <
lists@...> wrote:

test.asm:

SECTION AAA
org 0
SECTION BBB
SECTION CCC

SECTION AAA
defw __i_fcntl_fdstruct_1

SECTION BBB

__i_fcntl_fdstruct_1:
defb 205

SECTION CCC
defw __i_fcntl_fdstruct_1

defc __i_fcntl_fdstruct_2 = __i_fcntl_fdstruct_1

z80asm -b -o=test test.asm

test.map shows this:

__i_fcntl_fdstruct_1 = 0002, L: test
__i_fcntl_fdstruct_2 = 0003, L: test

The two labels are not equal despite the defc.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple
OSs.
http://pubads.g.doubleclick.net/gampad/ ... 1&iu=/4140

Post by **alvin** » Mon Dec 07, 2015 8:30 pm

The cause was that the DEFC expression was being computed in section CCC, but with terms refering to section BBB. The expression was computed at the compile phase with a wrong base address.
The problem should now be fixed. Any DEFC expression that depends on symbols from other modules or sections is now postponed to the link phase.

I'm getting a different error now:

Error at file 'test.asm' line 13: symbol '__i_fcntl_fdstruct_2' not defined
1 errors occurred during assembly

That's the line number corresponding to the defc defining that symbol.

z80asm -b -o=test test.asm

test.asm:

SECTION AAA
org 0
SECTION BBB
SECTION CCC

SECTION AAA
defw __i_fcntl_fdstruct_1

SECTION BBB

__i_fcntl_fdstruct_1:
defb 205

SECTION CCC
defw __i_fcntl_fdstruct_1

defc __i_fcntl_fdstruct_2 = __i_fcntl_fdstruct_1

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/ ... 1&iu=/4140

pscust · Post by **pscust** » Mon Dec 07, 2015 10:32 pm

Should be fixed now. I had added __i_fcntl_fdstruct_2 to the defw in the
test script, and this masked another bug.

Regards,
Paulo

On Mon, Dec 7, 2015 at 8:30 PM, alvin (alvin_albrecht@...) <
lists@...> wrote:

The cause was that the DEFC expression was being computed in section CCC,
but with terms refering to section BBB. The expression was computed at the
compile phase with a wrong base address.
The problem should now be fixed. Any DEFC expression that depends on
symbols from other modules or sections is now postponed to the link phase.
I'm getting a different error now:

Error at file 'test.asm' line 13: symbol '__i_fcntl_fdstruct_2' not defined
1 errors occurred during assembly

That's the line number corresponding to the defc defining that symbol.

z80asm -b -o=test test.asm

test.asm:

SECTION AAA
org 0
SECTION BBB
SECTION CCC

SECTION AAA
defw __i_fcntl_fdstruct_1

SECTION BBB

__i_fcntl_fdstruct_1:
defb 205

SECTION CCC
defw __i_fcntl_fdstruct_1

defc __i_fcntl_fdstruct_2 = __i_fcntl_fdstruct_1

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple
OSs.
http://pubads.g.doubleclick.net/gampad/ ... 1&iu=/4140

Post by **alvin** » Thu Dec 10, 2015 5:15 am

Should be fixed now. I had added __i_fcntl_fdstruct_2 to the defw in the test script, and this masked another bug.

It seems to be working now thanks!

Another issue I have is with memory maps. It isn't a bug in the sense code won't compile as expected - it's a usage issue.

test.asm:

;;; here's a memory map

section main
org 0
section PART_A
section PART_B
section PART_C

;;; here's some data placed in sections

section PART_B
IN_B: defs 2

section PART_A
IN_A: defs 1

section PART_C
IN_C: defs 3

;; here we introduce a new section and try to place it inside the defined memory map

section PART_B
section APPEND_TO_B
IN_B2: defs 4

z80asm -b -o=test test.asm

The map file shows:

ASMTAIL_PART_A = 0001, G:
ASMTAIL_PART_B = 0003, G:
ASMTAIL_PART_C = 0006, G:
ASMTAIL_APPEND_TO_B = 000A, G:

In other words the section "APPEND_TO_B" is added after all previously appearing sections.

With this:

section PART_B
section APPEND_TO_B
IN_B2: defs 4

I am expecting the previously unmet section "APPEND_TO_B" to be inserted after section "PART_B".

Because this doesn't happen, third party libraries with their own section names cannot be used without hand-editing the supplied memory maps to have those sections put in the proper CODE, DATA or BSS area. If they are not hand edited in, they will be appended after the last section defined in the supplied memory map and that is the BSS section. Any third party code / data will become part of the BSS section which will be zeroed by the crt on startup.

What I wanted to do is supply general containers in the defined memory map like "code_lib", "rodata_lib", "data_lib", "bss_lib", etc and then third party libraries could just place themselves there by default by using code like this:

SECTION code_lib
SECTION code_gunzip

The advantage of using many sections, including ones for external libraries, is that using --split-bin you can see exactly what their memory cost is but also the user has the option of placing parts of library code in a memory map of his own making.

------------------------------------------------------------------------------

pscust · Post by **pscust** » Thu Dec 10, 2015 7:12 am

If I understand the problem, what you need is a section group (code, data,
bss) and the possibility to declare that a new section belongs to the end a
given group, something in the lines of:

group code, data, bss
section code_lib in code
section code_gunzip in code

Sections wihout a declared group would be added to a predefined group –
empty group? last used group?

This solution still does not solve the problem of having to hand-edit third
party libraries, adding the group of each section.

There may exist a “standard?? syntax of defining this kind of grouping
(instead of the “in?? keyword). The “in?? keyword seems out-of-line, and a
simple “,?? may be confusing. Do you know other assemblers from which the
syntax may be used?

In terms of implementation, it should be straightforward, as sections are
kept in a memory-map-order structure.

Did I understand the problem?

Does anybody have a syntax proposal?

Any further comments?

Regards,

Paulo

On Thu, Dec 10, 2015 at 5:15 AM, alvin (alvin_albrecht@...) <
lists@...> wrote:

Should be fixed now. I had added __i_fcntl_fdstruct_2 to the defw in the

test script, and this masked another bug.

It seems to be working now thanks!

Another issue I have is with memory maps. It isn't a bug in the sense

code won't compile as expected - it's a usage issue.

test.asm:

;;; here's a memory map

section main
org 0
section PART_A
section PART_B
section PART_C

;;; here's some data placed in sections

section PART_B
IN_B: defs 2

section PART_A
IN_A: defs 1

section PART_C
IN_C: defs 3

;; here we introduce a new section and try to place it inside the defined

memory map

section PART_B
section APPEND_TO_B
IN_B2: defs 4

z80asm -b -o=test test.asm

The map file shows:

ASMTAIL_PART_A = 0001, G:
ASMTAIL_PART_B = 0003, G:
ASMTAIL_PART_C = 0006, G:
ASMTAIL_APPEND_TO_B = 000A, G:

In other words the section "APPEND_TO_B" is added after all previously

appearing sections.

With this:

section PART_B
section APPEND_TO_B
IN_B2: defs 4

I am expecting the previously unmet section "APPEND_TO_B" to be inserted

after section "PART_B".

Because this doesn't happen, third party libraries with their own section

names cannot be used without hand-editing the supplied memory maps to have
those sections put in the proper CODE, DATA or BSS area. If they are not
hand edited in, they will be appended after the last section defined in the
supplied memory map and that is the BSS section. Any third party code /
data will become part of the BSS section which will be zeroed by the crt on
startup.

What I wanted to do is supply general containers in the defined memory

map like "code_lib", "rodata_lib", "data_lib", "bss_lib", etc and then
third party libraries could just place themselves there by default by using
code like this:

SECTION code_lib
SECTION code_gunzip

The advantage of using many sections, including ones for external

libraries, is that using --split-bin you can see exactly what their memory
cost is but also the user has the option of placing parts of library code
in a memory map of his own making.

------------------------------------------------------------------------------

Post by **alvin** » Thu Dec 10, 2015 6:33 pm

If I understand the problem, what you need is a section group (code, data, bss) and the possibility to declare that a new section belongs to the end a given group, something in the lines of:
group code, data, bss
section code_lib in code
section code_gunzip in code
Sections wihout a declared group would be added to a predefined group ? empty group? last used group?

Here's what I am doing now:

SECTION CODE
org start_address
section code_clib
section code_lib
section code_compiler
....
SECTION CODE_END
SECTION DATA
(optional org)
...
SECTION DATA_END
SECTION BSS
(optional org)
...
SECTION BSS_END

The org will optionally separate the CODE, DATA, BSS into individual binaries on output. The "CODE" section spans ASMHEAD_CODE to ASMHEAD_CODE_END and similarly with the DATA and BSS sections. So the grouping is effectively done with a "begin" section (CODE) and an "end" section (CODE_END) bracketing a block.

What's needed to complete this is a way to insert new sections into those blocks and the method fitting into how things work now is as mentioned:

SECTION code_lib
SECTION code_gunzip

If the memory map is defined by a linked list of sections then the first time "code_gunzip" is seen, it could be inserted into the memory map right after "code_lib".

With grouping I think we will eventually need that too but even with a grouping ability, I still think the above is useful.

There are a couple of other needs that may come up. One is the memory map may be fragmented so in order to make maximum use of the available memory, it would be good if the linker could flow code and data around defined holes.

So, eg, suppose I have ram from 16384 to 24575, then a display file, then ram from 31488 to 65535, I may want to say there's a section extending from 16384 to 65535 but there's also a hole in it from 24576 to 31487 so please flow around it. This couldn't be done automatically without a grouping feature. There are also systems with different kinds of ram. The zx81, eg, has ram that cannot contain executable code which means we'd have to assign different groups to those different ram blocks and enforce a size on it. So a group may need to have a size attribute. To complete things you may want to allow code and data to flow from one group to another if the first is filled.

Those are just things in my head right now. I haven't investigated other assemblers in detail with regard to these features but, at least superficially, I didn't really like the way they handled things. We could probably do some research and think about what we need.

------------------------------------------------------------------------------

pscust · Post by **pscust** » Sun Dec 13, 2015 1:45 am

I'm thinking of something in these lines, although I am not fond of the
"before" and "after" keywords in the syntax:

section aaa ; appends a section at the end of the list
section aaa1 after aaa ; inserts a section imediatelly after aaa
section aaa2 before aaa ; inserts a section imediatelly before aaa

The change will not be simple, as it impacts the way addresses are
currently computed. Please let me know if this solves the requirement
before I start digging in the code.

Any suggestion on the syntax or functionality?

Regards,
Paulo

On Thu, Dec 10, 2015 at 6:33 PM, alvin (alvin_albrecht@...) <
lists@...> wrote:

If I understand the problem, what you need is a section group (code,
data, bss) and the possibility to declare that a new section belongs to the
end a given group, something in the lines of:
group code, data, bss
section code_lib in code
section code_gunzip in code
Sections wihout a declared group would be added to a predefined group –
empty group? last used group?
Here's what I am doing now:

SECTION CODE
org start_address
section code_clib
section code_lib
section code_compiler
....
SECTION CODE_END
SECTION DATA
(optional org)
...
SECTION DATA_END
SECTION BSS
(optional org)
...
SECTION BSS_END

The org will optionally separate the CODE, DATA, BSS into individual
binaries on output. The "CODE" section spans ASMHEAD_CODE to
ASMHEAD_CODE_END and similarly with the DATA and BSS sections. So the
grouping is effectively done with a "begin" section (CODE) and an "end"
section (CODE_END) bracketing a block.

What's needed to complete this is a way to insert new sections into those
blocks and the method fitting into how things work now is as mentioned:

SECTION code_lib
SECTION code_gunzip

If the memory map is defined by a linked list of sections then the first
time "code_gunzip" is seen, it could be inserted into the memory map right
after "code_lib".

With grouping I think we will eventually need that too but even with a
grouping ability, I still think the above is useful.

There are a couple of other needs that may come up. One is the memory map
may be fragmented so in order to make maximum use of the available memory,
it would be good if the linker could flow code and data around defined
holes.

So, eg, suppose I have ram from 16384 to 24575, then a display file, then
ram from 31488 to 65535, I may want to say there's a section extending from
16384 to 65535 but there's also a hole in it from 24576 to 31487 so please
flow around it. This couldn't be done automatically without a grouping
feature. There are also systems with different kinds of ram. The zx81,
eg, has ram that cannot contain executable code which means we'd have to
assign different groups to those different ram blocks and enforce a size on
it. So a group may need to have a size attribute. To complete things you
may want to allow code and data to flow from one group to another if the
first is filled.

Those are just things in my head right now. I haven't investigated other
assemblers in detail with regard to these features but, at least
superficially, I didn't really like the way they handled things. We could
probably do some research and think about what we need.

------------------------------------------------------------------------------

Post by **alvin** » Mon Dec 14, 2015 10:19 pm

I'm thinking of something in these lines, although I am not fond of the "before" and "after" keywords in the syntax:

section aaa ; appends a section at the end of the list
section aaa1 after aaa ; inserts a section imediatelly after aaa
section aaa2 before aaa ; inserts a section imediatelly before aaa

The change will not be simple, as it impacts the way addresses are currently computed. Please let me know if this solves the requirement before I start digging in the code.

Any suggestion on the syntax or functionality?

Posting here just to make sure it is reflected to everyone.

I'm still thinking about it and will comment after the release is done.

------------------------------------------------------------------------------