Spark Logic

Debugging using openocd with arm-none-eabi-gdb

Rick Kimball Wed Jul 18, 2018 6:05 pm

I’ve been working on a custom Arduino environment and my latest focus is automating upload and debugging using openocd with an stlink-v2 on linux. I’ve attached my arduino-debugger bash script. In my case, it is invoked from my custom platform.txt and boards.txt . However, my script might also be useful to the rest of you. It can be invoked from the command line in combination with a running Arduino IDE.
update: Alternatively, you can take a look at the attached zip file for use with Roger’s libmaple core, directly from the Arduino IDE.

Assuming I’m testing a script called working_leds. I’ve compiled it and uploaded it to my bluepill and the Arduino is still running. I looked over in /tmp and see that the build directory is /tmp/arduino_build_14749, here is how I invoke it to do debug only:

$ ./arduino-debugger -d1 /tmp/arduino_build_14749 working_leds.ino no_upload

Here is what it looks like on my desktop:
Image

Note: This script assumes that the openocd and arm-none-eabi-gdb executables are in your path.
file:arduino-debugger
#!/bin/bash
# File: arduino-debugger - upload and auto-launch debugger
# Desc: linux specific uploader and debugger using openocd and arm-none-eabi-gdb
# note: this might work on Mac OS/X (not tested)
#
# Author: Rick Kimball
#
#set -x
export ARG_CNT="$#"

if [ ${ARG_CNT} -lt 3 ]; then
echo "usage: $0 [-d0|-d1] arduino_build_tmp_path sketch [no_upload]"
echo " debug flag - openocd debug flags [-d0 silent|-d1 verbose]"
echo " arduino_build_tmp_path - the temp /tmp/arduino_build_nnnnn directory"
echo " sketch - the sketch name .. blink.ino"
echo " [no_upload] - optionally skip upload, just debug"
echo " example:"
echo " $0 -d1 /tmp/arduino_build_433659 blink.ino"
exit 1
fi

export TOOLS_DIR=$(dirname $0)
export DEBUG_FLG=${1}
export BUILD_DIR=${2}
export SKETCH=${3}
export BIN_FILE=${BUILD_DIR}/${3}.bin
export HEX_FILE=${BUILD_DIR}/${3}.hex
export ELF_FILE=${BUILD_DIR}/${3}.elf

#-----------------------------------------------------------------------
# if you define OPENOCD_TARGET we use it, if not we default to stm32f1x.cfg
if [ -z ${OPENOCD_TARGET+x} ]; then
export OPENOCD_TARGET=stm32f1x.cfg
fi

#----------------------------------------------------------------------------
# if openocd is hanging around, kill it
killall openocd >/dev/null 2>&1

set -e # worry about errors and exit from now on

#----------------------------------------------------------------------------
cd "${BUILD_DIR}"

#----------------------------------------------------------------------------
# upload to the board using the .bin or .hex file
if [ ${ARG_CNT} -eq 3 ]; then
if [ -f ${BIN_FILE} ]; then
openocd ${DEBUG_FLG} -f interface/stlink.cfg -f target/${OPENOCD_TARGET} \
-c "program ${BIN_FILE} verify reset exit 0x08000000"
else
openocd ${DEBUG_FLG} -f interface/stlink.cfg -f target/${OPENOCD_TARGET} \
-c "program ${HEX_FILE} verify reset exit"
fi
else
echo "skipping upload will use firmware already on flash"
fi

#----------------------------------------------------------------------------
# start up openocd server in the background minimized
xterm -geometry 80x43 -iconic \
-T 'Arduino GDB Server' \
-e openocd -f interface/stlink.cfg -f target/${OPENOCD_TARGET} &

#----------------------------------------------------------------------------
# start up the arm-none-eabi-gdb ui and wait for it to exit
xterm -geometry 100x45 -bg \#ffffe0 -fg \#000000 \
-T 'Arduino Debugger' \
-e arm-none-eabi-gdb \
-n \
-ex 'set print repeats 1024' \
-ex 'set pagination off' \
-ex 'set confirm off' \
-ex 'set mem inaccesible-by-default off' \
-ex "source ${TOOLS_DIR}/armv7m-vecstate.gdb" \
-ex 'target remote :3333' \
-ex 'mon reset init' \
-ex 'tbreak setup' \
-ex 'layout split' \
-ex 'continue' \
"${ELF_FILE}"

set +e
#----------------------------------------------------------------------------
# if anything went wrong this will kill the openocd gdb server
killall openocd >/dev/null 2>&1

exit 0

Replies

Rick Kimball Wed Jul 18, 2018 6:57 pm

Here is an example of debugging a HardFault caused by the sketch accessing an invalid memory address. To my code, I added the line:

volatile uint8_t crashme = *(volatile uint8_t *)0x30000000; // invalid address

This is bad code, it is trying to access 0x30000000 which is an invalid address. Eventually, It is going to end up in my default exception handler (which is just a forever loop). Normally you would get into the exception handler and not know why it faulted. Because I automatically include the vectstate.gdb script in the arduino-debugger, you can call this custom gdb command and find out which line of code caused your issue.

Here it is what it looks like just before the code invokes the bad access instruction:

Image

After I continued with the gdb ‘c’ command, it got stuck and stopped working. In the gdb command line, I pressed CTRL+C to interrupt execution, and then typed the gdb command ‘vecstate’ to see what caused the hard fault:
Image

The code was in a busy loop because of the HardFault exception handler being called. The busy loop is this instruction:

0x80003a0 <Reset_Handler+48> b.n 0x80003a0 <Reset_Handler+48>

The vecstate command walks the stack and queries the system registers to figure out where it came from and which exception was triggered. You can see that it identifies working_leds.ino, line 55 in as the source of the crash … pretty nice!

fpiSTM Wed Jul 18, 2018 8:24 pm

Thanks Rick. :!:

michael_l Thu Jul 19, 2018 7:37 pm

Rick, this is very cool !

Vassilis Fri Jul 20, 2018 1:45 pm

Cool !

Rick Kimball Sun Jul 22, 2018 6:49 pm

In the first post of this thread I added a zip file that contains changes and my arduino-debugger script for use with Roger’s libmaple core. It adds another upload method ‘openocd+debug’ menu. The menu item launches the arduino-debugger script from within the Arduino IDE. It will upload the code using openocd and then start a debug session.

I’ve only added this upload method to the genericSTM32F103C board (AKA bluepill board). If you want to use it with other boards you can copy pasta what I did with the boards.txt file. Also, I’ve only tested with linux, it will probably work for OS/X but has not been tested. However it definitively won’t work for Windows.

Image

I tested it out on the ASCIITable sketch. After you press upload, the arduino-debugger runs until you exit. Because of this, you can’t use the Arduino Serial Monitor to view the native USB output. This is easily solved by launching a putty serial terminal session after the setup() function is started. Waiting till setup() is run allows the native USB to enumerate and create a /dev/ttyACM0 you can open in putty. Once putty is running you can issue debug commands and watch the serial output at the same time.
Image

stevestrong Tue Jul 24, 2018 9:16 am

Rick, thanks for this nice how-to.

Did you try to debug the F4 chip, too?

Rick Kimball Tue Jul 24, 2018 11:36 am

I haven’t stevestrong. You would have to change the openocd_target script to stm32f4x.cfg for that to work.

stevestrong Tue Jul 24, 2018 12:16 pm

I remember I had problems debugging the F4 chip when stepping through the code, the memory window showed alternatively different or invalid values for a specific memory range (I think it was the code area). That’s why I’m asking, whether you also encountered some issues for F4 or not. And of course I used a cfg for F4.

Rick Kimball Tue Jul 24, 2018 12:25 pm

Which debugger were you using? Was the code compiled for debug or optimization?

In general, optimized code does follow an linear path through the code. You need to look at the c code, the asm code, register values and dump memory to follow what is happening. Hence, the reason I open with the gdb split layout.

stevestrong Tue Jul 24, 2018 1:57 pm

I used Eclipse (Sloeber) + OpenOCD, with debug info.
Nwevertheless, I think whatever optimization one use, the code area should not show different or invalid values when stepping through. :?

Rick Kimball Tue Jul 24, 2018 3:52 pm

I tend to compile for size optimization so I’m used to the erratic c source single stepping. If you step through at the asm level it seems to produce the results you expect.

optimized C code doesn’t not equate one to one for the asm produced. A C statement when optimized may be using the code in another statement. As you step through, you may actually leave the current line and go to another line in the same function.

I tend to use the gdb nexti or (ni) and stepi or (si) instructions to step through. Here are often used commands i use in gdb

gdb commands wrote:(gdb) help nexti
Step one instruction, but proceed through subroutine calls.
Usage: nexti [N] Argument N means step N times (or till program stops for another reason).
(gdb) help stepi
Step one instruction exactly.
Usage: stepi [N] Argument N means step N times (or till program stops for another reason).
(gdb) help info registers
List of integer registers and their contents, for selected stack frame.
Register name as argument means describe only that register.
(gdb) help x
Examine memory: x/FMT ADDRESS.
ADDRESS is an expression for the memory address to examine.
FMT is a repeat count followed by a format letter and a size letter.
Format letters are o(octal), x(hex), d(decimal), u(unsigned decimal),
t(binary), f(float), a(address), i(instruction), c(char), s(string)
and z(hex, zero padded on the left).
Size letters are b(byte), h(halfword), w(word), g(giant, 8 bytes).
The specified number of objects of the specified size are printed
according to the format.

Defaults for format and size letters are those previously used.
Default count is 1. Default address is following last thing printed
with this command or “print”.

stevestrong Tue Jul 24, 2018 5:16 pm

I am talking about a memory window with a fixed address, it alternates between showing good values / all 0 / non-sense values for consecutive steps.

You say that it must be the refresh process of Eclipse? Hm, for blue pill it did not happen.

Rick Kimball Tue Jul 24, 2018 6:26 pm

I don’t have sloeber installed but I gave the Atollic debugger a try and left the memory browser window open. Maybe you are seeing the memory window changing and display garbage as it is running.

The only time the memory window is valid and will refresh is when you hit a break point or when you interrupt execution. At other times it won’t update the display. When ever I hit a break point I see the memory browser update. I noticed at least in atollic, if the memory I was watching changed the color changed to red for the memory bytes that changed as I single stepped through the code.

However, this is still part of my point about graphical debuggers. When the breakpoint is hit, the eclipse UI has to query the gdb server and refresh all the values in the windows that are visible.

After I played with this some more, I noticed there is a way to control the update frequency of the memory browser. You might check to see if sloeber has this option also:
Image

I noticed that if you select “Always” as you single step if the memory changes it will update. If you have “On Breakpoint” selected it doesn’t update as you single step but it will update the value that was contained at the time of the breakpoint. If you select “Manual” if never updates unless you right click select the “Refresh” button or press the “GO” button.

Sign in to reply