Differences

This shows you the differences between two versions of the page.

--- etc:users:jcmvbkbc:linux-xtensa:esp32s3 [2023/05/08 03:05] – jcmvbkbc
+++ etc:users:jcmvbkbc:linux-xtensa:esp32s3 [2024/03/15 21:07] (current) – jcmvbkbc
@@ Line 2: / Line 2: @@
 Sources:
+  * https://github.com/jcmvbkbc/xtensa-dynconfig/tree/original
+  * https://github.com/jcmvbkbc/config-esp32s3
   * https://github.com/jcmvbkbc/esp-idf/tree/linux-5.0.1
-  * https://github.com/jcmvbkbc/linux-xtensa/tree/xtensa-6.4-esp32
+  * https://github.com/jcmvbkbc/linux-xtensa/tree/xtensa-6.7-esp32
-  * https://github.com/jcmvbkbc/binutils-gdb-xtensa/tree/xtensa-2.40-fdpic
+  * https://github.com/jcmvbkbc/binutils-gdb-xtensa/tree/xtensa-2.41-fdpic
-  * https://github.com/jcmvbkbc/gcc-xtensa/tree/xtensa-14-fdpic
+  * https://github.com/jcmvbkbc/gcc-xtensa/tree/xtensa-14-4992-fdpic
-  * https://github.com/jcmvbkbc/uclibc-ng-xtensa/tree/xtensa-fdpic
+  * https://github.com/jcmvbkbc/uclibc-ng-xtensa/tree/xtensa-1.0.44-fdpic
-  * https://github.com/jcmvbkbc/buildroot/tree/xtensa-2023.02-fdpic
+  * https://github.com/jcmvbkbc/buildroot/tree/xtensa-2023.11-fdpic
+  * https://github.com/jcmvbkbc/crosstool-NG/tree/xtensa-fdpic
-Apply mforce-l32 patch on top of gcc.
+Scripts with all steps below: https://github.com/jcmvbkbc/esp32-linux-build
+===== Details =====
+  * [[.:esp32s3:flash]]
+  * [[.:esp32s3:gpio]]
+  * [[.:esp32s3:linux-ipc]]
+  * [[.:esp32s3:mmu]]
+  * [[.:esp32s3:wifi]]
+===== Things that work =====
+  * WiFi. Use the script that builds firmware based on esp-hosted. It runs on core 0, linux runs on core 1, special linux IPC is used for communication. WiFi transport that uses linux IPC is added both to the firmware and to the linux kernel wifi driver. Not all wifi security options may be working, e.g. open and wpa2-psk are working and 802.11w is not.
+  * Writing to FLASH and using ESP FLASH partition table. Driver based on linux IPC sends FLASH-related requests to the firmware. Default configuration has an etc partition that is flashed with /etc file system and mounted at boot time. The file system is writable and it can be used to store things like wpa_supplicant.conf, /etc/passwd, /etc/shadow, ...
+  * USB serial. It is visible as the /dev/ttyACM1 (in kernels based on v6.5) or the /dev/ttyGS3 (in kernels based on v6.6-rc and newer) device inside the linux environment. (The change of name was requested by the linux TTY subsystem maintainer, the change of number is to keep UART ports numbering coherent with esp32s3 TRM).
+  * GPIO, including interrupts. Software I2C over GPIO and software SPI over GPIO.
+  * hardware SPI. Tested clock speed up to 20MHz with SD card.
+  * clock frequency detection. CPU, XTAL and APB clocks may be preset by the bootloader, the kernel will understand and use preset frequencies without additional configuration.
+  * ssh server and ssh client. There's an issue using scp from openssh version 9 because that version switched from the original scp protocol to sftp. The option -O makes it use the now legacy scp.
+  * mounting NFS shares.
+  * running executable code from outside the rootfs, e.g. from /etc or from NFS mounts. A special hack in the kernel code does address remapping, so that when an executable memory mapping is created, the mapping address points to the PSRAM alias in the IRAM address range. This does not work for ESP32 as it doesn't have executable mappings of PSRAM.
+  * passing command line from bootloader to the kernel. Bootloader reads the file /etc/cmdline if it exists and passes its contents to the kernel as the command line. The bootloader does JFFS2 file system parsing for that. E.g. the following line can be used to move console to the USB serial: <code>earlycon=esp32s3acm,mmio32,0x60038000 console=ttyGS3 debug rw root=mtd:rootfs no_hash_pointers</code>
+  * strace (a one-line fix is needed for the mainline strace to correctly handle the initial exec).
+  * perf stat (heavy patching is needed to build it for nommu). -D1 is needed to properly enable events (perf relies on ability to run code after the fork but before the exec in the child process to manage events on systems with mmu, -D1 looks like a good workaround for nommu case).
+  * c++ exceptions, c cleanup routines, forced stack unwinding, unwinding over signal frames.
+  * TLS and NPTL. Some corner cases still need attention though.
+===== Things that don't work =====
+  * about 290 failing tests in the gcc testsuite, most related to TLS and linuxthreads limitations. About 300 failing tests in the g++ testsuite.
+  * mmap with MAP_FIXED flag. By design of the nommu linux, but it seems to me that it doesn't have to be like that. It's usually not a big deal, but that's the reason there's no module information in the /proc/*/maps other than for the ld.so and the executable.
+  * tcpdump and libpcap in general. It tries to mmap the packet socket and it's missing a few things (mm/nommu.c doesn't know what capabilities to assign to S_IFSOCK files, the socket file needs to have get_unmapped_area callback defined), but most importantly the packet_mmap code heavily relies on the presence of mmu. Although it seems that it could be worked around.
+  * bluetooth. It requires heavy userspace support that includes dbus which wants fork. Looks like it can be worked around, but the amount of bloat is discouraging.
+  * perf record. It requires mmapping perf events and the kernel code somewhat relies on the presence of mmu. I've tried to add a workaround, but something is still missing.
+===== Building/flashing/running =====
 <code>
-$ ( cd build-xtensa-fdpic-toolchain-esp32s3 ; rm -rf * ; FLAGS_FOR_TARGET='-mtext-section-literals -mfdpic -O2 -g -mforce-l32' nice ../build-xtensa-fdpic-toolchain.sh )
+$ git clone https://github.com/jcmvbkbc/esp32-linux-build
-$ ( cd build-xtensa-fdpic-nothread-esp32s3 ; rm -rf * ; CROSS_COMPILE=`pwd`/../build-xtensa-fdpic-toolchain-esp32s3/root/bin/xtensa-linux-uclibcfdpic- TARGET_CFLAGS='-mauto-litpools -mforce-l32 -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64' ../build.sh )
+$ cd esp32-linux-build
-$ ln -s `pwd`/build-xtensa-fdpic-nothread-esp32s3/sysroot build-xtensa-fdpic-toolchain-esp32s3/root/xtensa-linux-uclibcfdpic/
+$ ./rebuild-esp32s3-linux-wifi.sh
 </code>
+It will run the following steps to build the toolchain, the kernel, the root and etc filesystem images and the firmware and it will flash it.
+==== Build toolchain dynconfig library and export XTENSA_GNU_CONFIG for use by the toolchain ====
 <code>
-$ nice make -C buildroot O=`pwd`/build-xtensa-2023.02-fdpic-esp32s3 qemu_xtensa_kc705_xip_defconfig
+$ git clone https://github.com/jcmvbkbc/xtensa-dynconfig -b original
-$ # adjust external toolchain location to the one built above
+$ git clone https://github.com/jcmvbkbc/config-esp32s3 esp32s3
-$ nice make -C buildroot O=`pwd`/build-xtensa-2023.02-fdpic-esp32s3
+$ make -C xtensa-dynconfig ORIG=1 CONF_DIR=`pwd` esp32s3.so
+$ export XTENSA_GNU_CONFIG=`pwd`/xtensa-dynconfig/esp32s3.so # make sure this environment variable is set for all commands involving building or using the toolchain
 </code>
+==== Build the toolchain ====
+<code>
+$ git clone https://github.com/jcmvbkbc/crosstool-NG.git -b xtensa-fdpic
+$ pushd crosstool-NG
+$ ./bootstrap && ./configure --enable-local && make
+$ ./ct-ng xtensa-esp32s3-linux-uclibcfdpic
+$ CT_PREFIX=`pwd`/builds nice ./ct-ng build
+$ popd
+</code>
+It doesn't complete successfully ATM, failing with the following message, but it builds and installs everything that's important.
+<code>
+...
+[INFO ]  Installing final gcc compiler: done in 572.47s (at 29:47)
+[INFO ]  =================================================================
+[INFO ]  Checking dynamic linker symlinks
+[EXTRA]    Checking dynamic linker for multilib ''
+[ERROR]    collect2: error: ld returned 1 exit status
+[ERROR]
+[ERROR]  >>
+[ERROR]  >>  Build failed in step 'Checking dynamic linker symlinks'
+[ERROR]  >>        called in step '(top-level)'
+[ERROR]  >>
+[ERROR]  >>  Error happened in: CT_DoExecLog[scripts/functions@377]
+[ERROR]  >>        called from: CT__FixupLDSO[scripts/functions@1695]
+[ERROR]  >>        called from: CT_IterateMultilibs[scripts/functions@1608]
+[ERROR]  >>        called from: CT_MultilibFixupLDSO[scripts/functions@1761]
+[ERROR]  >>        called from: uClibc_ng_post_cc[scripts/build/libc/uClibc-ng.sh@335]
+[ERROR]  >>        called from: do_libc_post_cc[scripts/build/libc.sh@38]
+[ERROR]  >>        called from: main[scripts/crosstool-NG.sh@697]
+[ERROR]  >>
+...
+</code>
+==== Build the rootfs and kernel image ====
+<code>
+$ git clone https://github.com/jcmvbkbc/buildroot -b xtensa-2023.08-fdpic
+$ nice make -C buildroot O=`pwd`/build-buildroot-esp32s3 esp32s3_defconfig
+$ buildroot/utils/config --file build-buildroot-esp32s3/.config --set-str TOOLCHAIN_EXTERNAL_PATH `pwd`/crosstool-NG/builds/xtensa-esp32s3-linux-uclibcfdpic
+$ buildroot/utils/config --file build-buildroot-esp32s3/.config --set-str TOOLCHAIN_EXTERNAL_PREFIX '$(ARCH)-esp32s3-linux-uclibcfdpic'
+$ buildroot/utils/config --file build-buildroot-esp32s3/.config --set-str TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX '$(ARCH)-esp32s3-linux-uclibcfdpic'
+$ nice make -C buildroot O=`pwd`/build-buildroot-esp32s3
+</code>
+==== Build and flash the bootloader, flash kernel and rootfs images ====
 <code>
+$ git clone https://github.com/jcmvbkbc/esp-idf -b linux-5.0.1
+$ pushd esp-idf
+$ . export.sh
 $ cd examples/get-started/linux_boot
+$ idf.py set-target esp32s3
+$ cp sdkconfig.defaults.esp32s3 sdkconfig
 $ idf.py build
 $ idf.py flash
-$ parttool.py -p /dev/ttyUSB0 write_partition --partition-name linux --input xip-esp32s3-esp32s3/arch/xtensa/boot/xipImage
+$ popd
-$ parttool.py -p /dev/ttyUSB0 write_partition --partition-name rootfs --input build-xtensa-2023.02-fdpic-esp32s3/images/rootfs.cramfs
+$ parttool.py write_partition --partition-name linux  --input build-buildroot-esp32s3/images/xipImage
+$ parttool.py write_partition --partition-name rootfs --input build-buildroot-esp32s3/images/rootfs.cramfs
+$ parttool.py write_partition --partition-name etc    --input build-buildroot-esp32s3/images/etc.jffs2
 </code>
-Gives the following:
+==== The result ====
 <code>
@@ Line 214: / Line 312: @@
 dcache size     : 0
 dcache flags    :
+~ # free
+              total        used        free      shared  buff/cache   available
+Mem:           7344        3264        3444           0         636        3208
 ~ #
 </code>