Mi­cro­con­trollers

Rasp­berry Pi

Freezing

System sta­bility with the Rasp­berry Pi has been a struggle for me. It can re­ally ruin the fun of dis­cov­ering mi­cro­con­trollers/​SBCs if you're con­stantly re­flashing SDs and swap­ping power sup­plies.

Once in a while, pretty much anyone who tin­kers with the Rasp­berry Pi may find that their system starts freezing up. It can be a total night­mare to debug. If you're dealing with this, first of all— you have my sym­pa­thies.

Now, let's get down to brass tacks and figure out what's causing this issue. A rasp­berry pi that is sud­denly locking up and re­quires a hard re­boot (power cycle), usu­ally suf­fers from one of the fol­lowing is­sues:

Power

Make sure you have an of­fi­cially-sup­ported power supply. A good one for the Rasp­berry Pi 4 (where I most often en­counter this problem) needs to have 5V and at least 3A of cur­rent.

If you have other pe­riph­erals (like a shield, USB de­vices, etc.), try dis­con­necting them and using the Pi for a while. If you're on a cir­cuit in your house with other power-hungry de­vices, try moving the Pi to an­other plug.

Tem­per­a­ture

If an RPi gets past a cer­tain tem­per­a­ture limit, it might freeze up and re­quire a power cycle— which could po­ten­tially cor­rupt your mi­croSD card! If your pi boots at all, look at the tem­per­a­ture:

If your tem­per­a­ture is con­sis­tently spiking above 70 or 80°C, you might con­sider a better cooling so­lu­tion— make sure you have a solid thermal con­nec­tion with any heatsinks, try adding a fan or larger heatsink. En­sure that your pi isn't closed off from am­bient air­flow (like in a sealed en­clo­sure).

CPU/​GPU Over­clocking

This is an easy one to debug, be­cause you can just re­move the over­clocking from you /boot/config.txt and see if the lock-up oc­curs. Often, this issue pig­gy­backs on tem­per­a­ture/​power is­sues.

To check and en­sure your Rasp­berry Pi has no over­clocking set­tings, in­spect the boot con­fig­u­ra­tion file:

If any of these are pre­sent and set above de­fault values, the Pi is over­clocked. To re­move over­clocking, com­ment them out with # or delete those lines en­tirely.

Un­sup­ported GPU Dri­vers

One of my Rasp­berry Pi SBCs has En­deav­ourOS in­stalled. This is an awe­some op­tion if you want an easy Arch Linux in­stall on the Pi. I don't know if it's doc­u­mented, but I kept seeing system lock-ups when the RPi's v3d kernel module (Video­Core VI 3D ac­cel­er­a­tion driver) was en­abled. It could be un­re­lated to the OS, but here were the symp­toms:

• Random system freezes/​lockups re­quiring hard power cy­cles
• No kernel panic, no er­rors on screen— just an un­re­spon­sive system
• Pro­gres­sively shorter up­times (per­haps under a faulty PSU... after re­place­ment, less fre­quent, but still hap­pening)
• Cor­rup­tion on boot par­ti­tion (journal uncleanly shut down)

Root Causes

The v3d kernel module has a bug in the DRM GPU sched­uler on kernel 6.18.18. The failure se­quence:

1. A GPU bin job hangs
2. The timeout han­dler fires: v3d_bin_job_timedout
3. It calls v3d_gpu_reset_for_timeout -> drm_sched_stop
4. drm_sched_stop calls dma_fence_wait_timeout on a fence that never sig­nals
5. The kernel worker thread en­ters un­in­ten­tion­ally sleep (D state) per­ma­nently
6. Re­peated GPU re­sets flood the sched­uler, cas­cading into a full system lockup
7. No panic or oops is logged— the system simply be­comes un­re­spon­sive

This oc­curs re­gard­less of the dis­play overlay (both vc4-kms-v3d and vc4-fkms-v3d), be­cause v3d module loads in­de­pen­dently for 3D ac­cel­er­a­tion.

Key log ev­i­dence

Fixes

1. Black­list v3d module (pri­mary fix)

Then re­build initramfs:

2. Switch dis­play overlay to FKMS

This uses the firmware-backed dis­play path in­stead of full KMS (Kernel Mode Set­ting).

3. En­able hard­ware watchdog

Also, in /boot/config.txt:

4. Fix fstab for au­to­matic fsck

The root par­ti­tion was missing from fstab en­tirely, and the boot par­ti­tion had pass 0 (no fsck). Now both are checked au­to­mat­i­cally on boot.

Di­ag­nostic Tools

VNC

Rem­mina

Taken from the "Linux Con­sulting and Training" docs:

When trying to con­nect from your Linux ma­chine using Rem­mina to a Rasp­berry Pi run­ning Rasp­berry PI OS with Re­alVNC en­abled you get the error:

Un­known au­then­ti­ca­tion scheme from VNC server

Re­alVNC only sup­ports a few se­cu­rity schemes. Authentication=VncAuth seems to be the only scheme that al­lows di­rect con­nec­tions from VNC-com­pat­ible Viewer pro­jects from third par­ties. In order to change to VncAuth scheme in your Rasp­bian and set a pass­word to ac­cept con­nec­tions from Rem­mina VNC plugin, open a SSH ses­sion (or a ter­minal window) on the Rasp­berry and gen­erate your VNC pass­word with:

Now, edit the file /root/.vnc/config.d/vncserver-x11

and add the fol­lowing line at the end of the file:

Now your config file should look more or less like mine:

c3abbea3b003a0b231737c0541892d72 is the en­crypted ver­sion of "rasp­berry"; your line will likely be dif­ferent.

Even­tu­ally, restart the VNC server ser­vice with:

...and you are ready to con­nect to your Rasp­berry Pi using Rem­mina.

Im­ager

How do I open "Ad­vanced Set­tings" in Rasp­berry Pi Im­ager ap­pli­ca­tion?

• Hold Ctrl+Shift+x on the main Im­ager screen