Moving to ARM (Part 2 – Booting the board)

(Continued from Part 1)

When I first purchased the Macchiatobin, I was under the impression that they had an OS image installed on the eMMC. Boy, was I in for a surprise. Booting the board with the serial console connected gives you something like this…

BootROM - 2.03
Starting CP-1 IOROM 1.07
Booting from SPI NOR flash 1 (0x32)
Found valid image at boot postion 0x000
lNOTICE: Starting binary extension
NOTICE: Gathering DRAM information
mv_ddr: mv_ddr-armada-17.06.1-g47f4c8b (Sep 27 2017 - 11:55:40)
mv_ddr: completed successfully
NOTICE: Booting Trusted Firmware
NOTICE: BL1: v1.3(release):armada-17.06.2:297d68f
NOTICE: BL1: Built : 11:55:45, Sep 27 2017
NOTICE: BL1: Booting BL2
lNOTICE: BL2: v1.3(release):armada-17.06.2:297d68f
NOTICE: BL2: Built : 11:55:47, Sep 27 2017
BL2: Initiating SCP_BL2 transfer to SCP
NOTICE: Load image to AP MSS
NOTICE: Loading MSS image from address 0x4023000 Size 0x504c to MSS at 0xf0580000
NOTICE: Done
NOTICE: BL1: Booting BL31
lNOTICE: BL31: v1.3(release):armada-17.06.2:297d68f
NOTICE: BL31: Built : 11:55:49, Sep 27 2017
l

U-Boot 2017.03-armada-17.06.3-ga33ecb8 (Sep 27 2017 - 11:55:14 +0200)

Model: MACCHIATOBin-8040
Clock: CPU 1300 [MHz]
DDR 800 [MHz]
FABRIC 800 [MHz]
MSS 200 [MHz]
DRAM: 16 GiB
U-Boot DT blob at : 000000007fb12fb8
EEPROM configuration pattern not detected.
Comphy chip #0:
Comphy-0: PEX0
Comphy-1: PEX0
Comphy-2: PEX0
Comphy-3: PEX0
Comphy-4: SFI
Comphy-5: SATA1
Comphy chip #1:
Comphy-0: SGMII1 1.25 Gbps
Comphy-1: SATA0
Comphy-2: USB3_HOST0
Comphy-3: SATA1
Comphy-4: SFI
Comphy-5: SGMII2 3.125 Gbps
UTMI PHY 0 initialized to USB Host0
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq led only pmp fbss pio slum part sxs
SATA link 0 timeout.
SATA link 1 timeout.
AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq led only pmp fbss pio slum part sxs
PCIE-0: Link down
MMC: sdhci@6e0000: 0, sdhci@780000: 1
SF: Detected w25q32bv with page size 256 Bytes, erase size 4 KiB, total 4 MiB
Net: mdio_register: non unique device name 'ethernet@0'
eth0: mvpp2-0, eth1: mvpp2-3, eth2: mvpp2-4, eth3: mvpp2-5
Hit any key to stop autoboot: 0

So far so good, right? Unfortunately, the board did not boot, and it dumped me to a prompt. I started looking into U-Boot, and it seems pretty sensible… it also feels somewhat reminiscent of the Open Firmware that was used on PowerPC Macintoshes, or Sun SPARC systems (among other older UNIX bootloaders). I was able to look at the filesystem on the eMMC, and it was definitely blank. Well darn, what do I do now?

The board has a block of jumpers to choose how the board boots up. It’s a good thing… because I’ll need it later (foreshadowing). I thought I would need it now, but U-Boot can boot from the SD card as long as it can see it. I’ll touch more on that in part 3.

So I had a look online, and on their website, they have a kernel image listed, as well as a few distro options available for download. The U-Boot firmware is dated from 2017, the kernel had a date of 2018 (but is also from 2017), and the distro images are from 2015-2016. In the grand scheme of Linux things, this software is ridiculously old (even for 2021), but I didn’t want to stay on it forever. I also didn’t have a way to copy the images over to the eMMC, so my only option was to copy them to a SD card, and boot from that instead.

I was able to get buildroot installed on the eMMC, but if I recall correctly, I didn’t have what I needed to install Gentoo yet. Unfortunately this next part gets a little sketchy, because this was a few years ago now and I didn’t document it at the time.

The next plan of action was to install Gentoo to the eMMC itself. Only problem was that the eMMC was only 6GB after formatting (I believe the SPI boot option uses a part of the eMMC for its storage), so I resorted to using a pair of USB sticks to supplement the eMMC to get the install done. The portage tree takes 1GB on average (excluding downloaded package files), the kernel source takes a few GB as well when its all unpacked and built, so there was no way it was going to fit on the eMMC on top of the standard system packages.

Initially I wanted to use the eMMC as a “recovery partition” and install the live environment onto SSDs. This took a few days to say the least. If it wasn’t bad enough that the board is a 2GHZ quad-core ARM, using a USB stick like it’s a hard drive will lead to it slowing to a crawl (and possibly even hanging indefinitely… I wound up throwing out 4 USB sticks trying to install the other 2 boards). It would have also helped if I knew which devices I needed to add to the kernel, thankfully most of the kernel stuff has been mainlined.

Eventually I got to a point where I was ready to reboot into the newer 5.13 kernel booting off the eMMC…

mvebu-cp110-comphy: unsupported SMC call, try updating your firmware

and with that I had no eMMC/SD, no network, and no SATA, basically nothing. Well, back to the drawing board… and by drawing board, I mean booting off the SD card again.

In part 3, I will go over how to upgrade the firmware on the board, so we can get the eMMC and SD card working again (because I’ll have to do this a few times… even more foreshadowing).