Context
Our team is porting a custom RTOS implementation (originally for an old Intel architecture) targetting the Zynq-7000 CPUs, therefor our main IDE/buildsystem of choice is Vitis. We are on the last version of Vitis Unified IDE (2024.2). It was decent until we got to the point of porting the packages, which on our system represent ELFs that have been specially compiled for the RTOS so that the loader can recognise and execute them. The problem is that we are unable to find how to link the GCC library (for the armv7-a+vfp) to our package, which is unfortunate because even a simple thing like a division needs a call to __aeabi_uidiv...
What we know
Vitis creates 2 binaries, the FSBL.elf and our_project.elf and performing an objdump or nm on both shows that the gcc and standart c functions are inside and are all resolved. Now to know how exactly they were linked, is extreamly complex since Vitis has it's build-system shared between custom pyhton files, custom configuration files, the Lopper and CMakeLists. I did try tracing the processes with ProcMon but it didn't give me any more clues, except pointing me to the directories where the pre-compiled libraries and the generated xil libraries seem to be.
Our package compilation
Since our packages include calls to functions/global variables of the OS we use the -r flag to make them relocatable, and when the package is getting loaded the OS loader will perform the necessary rellocation. Here is a sample makefile for the package compilation. The ld is not interesting to show and simply has several special sections for our package recognition and an entry definition.
CC = arm-none-eabi-gcc
LD = arm-none-eabi-ld
READELF = arm-none-eabi-readelf
OBJDUMP = arm-none-eabi-objdump
PACKAGE = package_another
# https://developer.arm.com/documentation/101754/0623/armclang-Reference/armclang-Command-line-Options
#
# no-common: Avoid multiple definition of global variables in each translation unit
# fPIC: Generate position-independent code
# ffunction-sections & -fdata-sections: Allows linker to optimize section removal
# nthumb-interwork: Generate code that supports calling between ARM and Thumb instructions
# mpic-data-is-text-relative: Helps the linker to optimize PIC code
CFLAGS = -c \
-mfpu=vfpv3 \
-mfloat-abi=hard \
-mcpu=cortex-a9 \
-fno-common \
-fPIC \
-ffunction-sections \
-fdata-sections \
-mthumb-interwork \
-mpic-data-is-text-relative \
-O0 \
-Iinclude
# r: Generate relocatable output
# gc-sections: Remove unused sections
# build-id=none: Do not generate a build ID
LDFLAGS = -r \
--gc-sections \
--build-id=none \
--no-undefined
all: $(PACKAGE).elf readelf objdump
$(PACKAGE).o: $(PACKAGE).c
$(CC) $(CFLAGS) $< -o $@
$(PACKAGE).elf: $(PACKAGE).o $(PACKAGE).ld
$(LD) -T $(PACKAGE).ld $< -o $@ $(LDFLAGS)
readelf:
$(READELF) -a $(PACKAGE).elf > READELF
objdump:
$(OBJDUMP) -d $(PACKAGE).elf > OBJDUMP
.PHONY: all clean readelf
clean:
rm -f $(PACKAGE).o $(PACKAGE).elf b
What We have tried
Because of the -r flag the only way to partially link the libgcc and libc is with the --whole-archive. So since we are on v7 (armv7) and we want the hard FP this is the path we chose:
/mnt/c/Xilinx/Vitis/2024.2/gnu/aarch32/nt/gcc-arm-none-eabi/aarch32-xilinx-eabi/usr/lib/thumb/v7+fp/hard/arm-xilinxmllibv7fphard-eabi/13.3.0/libgcc.a
Therefor, adding this line to the LDFLAGS
--whole-archive /mnt/c/Xilinx/Vitis/2024.2/gnu/aarch32/nt/gcc-arm-none-eabi/aarch32-xilinx-eabi/usr/lib/thumb/v7+fp/hard/arm-xilinxmllibv7fphard-eabi/13.3.0/libgcc.a --no-whole-archive \
But of course for one this is an overkill (way to many symbols and we will probably only ever need at most a dozen), and two there are still many undefined symbols so we need to link something else?
Question 1
What approach should we take to have a minimal version of the libgcc linked to our package? Guessing that it's not crucial for it to be the same one that the RTOS has (since it is barely used anyway even by the RTOS kernel)...
Question 2
Should we try to get away from the Vitis IDE? The only thing really holding us connected to it is the build system, and the lopper utility that allows to regenerate the device tree even if the XSA changes, and that also updates the BSP. Ease of compilation is still kind of in place, however going further it seems to be posing more problems than giving solutions. I know that it can be scripted using python, and we were successful in implementing a project rebuilding functionality (for github source control). However the scripting features are quite limited and not super well explained in the documentation.
I found something that works for me pretty well! Basically just a way to find a local libgcc from the downloaded toolchain:
CC = arm-none-eabi-gcc
LD = arm-none-eabi-ld
READELF = arm-none-eabi-readelf
OBJDUMP = arm-none-eabi-objdump
PACKAGE = package_another
# Find the location of libgcc with the correct ABI
LIBGCC := $(shell $(CC) -mfpu=vfpv3 -mfloat-abi=hard -mcpu=cortex-a9 -print-libgcc-file-name)
# https://developer.arm.com/documentation/101754/0623/armclang-Reference/armclang-Command-line-Options
#
# no-common: Avoid multiple definition of global variables in each translation unit
# fPIC: Generate position-independent code
# ffunction-sections & -fdata-sections: Allows linker to optimize section removal
# nthumb-interwork: Generate code that supports calling between ARM and Thumb instructions
# mpic-data-is-text-relative: Helps the linker to optimize PIC code
CFLAGS = -c \
-mfpu=vfpv3 \
-mfloat-abi=hard \
-mcpu=cortex-a9 \
-fno-common \
-fPIC \
-ffunction-sections \
-fdata-sections \
-mthumb-interwork \
-mpic-data-is-text-relative \
-O0 \
-Iinclude
# r: Generate relocatable output
# gc-sections: Remove unused sections
# build-id=none: Do not generate a build ID
LDFLAGS = -r \
--gc-sections \
--build-id=none \
--no-undefined
all: $(PACKAGE).elf readelf objdump b
$(PACKAGE).o: $(PACKAGE).c
$(CC) $(CFLAGS) $< -o $@
$(PACKAGE).elf: $(PACKAGE).o $(PACKAGE).ld
$(LD) -T $(PACKAGE).ld $< -o $@ $(LDFLAGS) $(LIBGCC)
readelf:
$(READELF) -a $(PACKAGE).elf > READELF
objdump:
$(OBJDUMP) -d $(PACKAGE).elf > OBJDUMP
b: $(PACKAGE).elf
python3 to_bin.py $< > $@
.PHONY: all clean readelf
clean:
rm -f $(PACKAGE).o $(PACKAGE).elf b
Hope this can help anyone with a similar problem