This tutorial will walk you through the steps needed to start debugging the Linux Kernel with a setup using
gdb. If you want to know more about
qemu, check the Official Repo, definitely one of the best open source projects out there.
This Setup gives you the capability to remotely debug a
qemu instance emulating The Linux Kernel, you can also debug Kernel Modules with this setup but we will not get into it in this tutorial.
1→ Build your own Custom Kernel:
To start we need to build our own kernel to have the executable binary image
bzImage so we can emulate it with
qemu , and the Kernel File Image
vmlinux with all debug info we need, to use it with
gdb to trace through the Kernel Code. Both of these files are obtained when we compile the Linux Kernel from source.
Building the Kernel from scratch takes time AND space, so if you want to boot into your newly compiled kernel, when setting up your VM make sure to manually partition your disk and give
/boot partition not less than 10GB to be able to boot the newly build kernel. THIS IS VERY IMPORTANT!
Detailed Steps for Building the Linux Kernel can be found in kernelnewbies.org
# after cloning the kernel tree from git, cd into <kernel-dir> # copy your distro's kernel config file into the cloned <kernel-dir> cd <kernel-dir> cp /boot/config-`uname -r` <kernel-dir>/config # editing the custom kernel config # you can choose your same kernel config you have by executing make olddefconfig # enough for what we need # or you can tinker with the new config, enable different built-in modules ..etc make menuconfig
// Use navigation to select drivers you want to build, and hit save.
We can edit the config file itself using a text editor and enable enough kernel config for debugging, I will not debug Kernel Modules in this tutorial, so as I said I would not bother build them.
For what we need, make sure to enable these configs:
CONFIG_DEBUG_INFO=y CONFIG_GDB_SCRIPS=y CONFIG_DBUG_KERNEL=y
There are other compiler configs to be enabled, but that is enough for our purposes.
Save the changes and build the Kernel:
sudo make -j3 sudo make -j3 modules_install # if you want concurrency, make sure to use less cores than you have # specially if you're running a VM `nproc` # to know how many cores you have
After finishing, navigate to
<kernel-dir>/arch/x86/boot you'll find the compiled kernel executable image
bzImage this is what we need
qemu to boot, it is a BIG image.
Now we have our compiled kernel, we can now restart our system, boot in the new kernel and hop into the next step.
Making a RAM disk image:
Now before generating a RAM disk image, we will stop a second to understand who the Linux system boots, first we have the
bzImage → the kernel as an executable binary, yet in order for the kernel to boot it needs an initial root filesystem
initramfs to get stuff going and setup correctly initial binaries for the real system to work, like the
init process aka the parent of all processes , and a bunch of other important binaries in
/sbin , when the
initramfs is done executing crucial binaries as root, it then looks for the REAL root filesystem to mount and pivot the root to it, free itself from memory, and the full system boots. but if it did not find the REAL root filesystem, it will boot the kernel and throw you in a recovery shell to boot it manually, or figure out what went wrong.
Have you experienced this lately?? 😉
So we have three components for booting a Linux system:
→ Kernel Image:
→ RAM disk:
→ Root file system:
but since we are not interested into making a full blown Linux system, we can stop at
initramfs being the root filesystem, and not provide a root filesystem/device to
qemu . as we will see everything will work fine with just an
Making a RAM disk depends on your distro, since I'm running Debian for this setup, this can be done by:
mkinitramfs -o ramdisk.img file ramdisk.img # ramdisk.img: gzip compressed data ... original size 26352128
Now we have what we need, Let's move on to compile
qemu from source:
I like building big software from source, because I can choose a minimal setup for just what I need,
qemu is an emulation software and has a dozen of target systems to emulate, we don't need all that, we are just interested in a
x86_64 bit Linux system.
Detailed steps for building
qemu can be found here .
# after cloning the repo, cd into qemu git clone git://git.qemu-project.org/qemu.git cd qemu mkdir build && cd build # configure target system we need / and other options ../configure --target-list=x86_64-softmmu --enable-debug # build make -j3
qemu has tons of options to enable, like enabling usb passthrough by using compiler options like
-libusb and configuring the graphic and screens ...etc, we ONLY want to emulate and boot the Linux Kernel. So .. practicing Minimalism. 🤟🏻
Now we've successfully built
qemu and can use it by executing
3→ Setting up the Environment:
Booting the Linux Kernel in
As a starter let's test booting the kernel:
qemu to boot the Linux kernel it needs two parameters,
-kernel <path-to-kernel-bzImage> and
-initrd <path-to-ramdik.img> ,
-m 512 for memory and that is VERY enough.
qemu-system-x86_64 -kernel <kernel-dir>/arch/x86_64/boot/bzImage \ -initrd <path-to>/ramdisk.img \ - m 512
And as we see, we've been thrown to an
initramfs recovery shell with very limited functionality.
Now let's connect the
qemu instance with
gdb for remote debugging.
gdb is adopted everywhere, almost all important projects/software will have a
GDBStub for debugging, in our case this task is done simply by adding
-s to the
qemu-system-x86_64 -kernel <kernel-dir>/arch/x86_64/boot/bzImage \ -initrd <path-to>/ramdisk.img \ - m 512 -s
That might look like as if we did nothing, because the kernel booted exactly the same as before. but if we connected with
gdb to port
qemu uses this port for
gdb remote debugging.
We see that
gdb connected successfully, but we couldn't catch anything. and if we closed the
gdb will complain about a remote communication error.
so we need to tell
qemu to STOP booting until connected with
gdb , simple as in a
-S option to add to the
qemu-system-x86_64 -kernel <kernel-dir>/arch/x86_64/boot/bzImage \ -initrd <path-to>/ramdisk.img \ - m 512 -s -S
Now we have the kernel waiting for us to connect remotely over port
Let's make it cooler and redirect the
qemu output to the main console window with a serial port terminal by adding a
-nographic option to not view the
qemu window and
-append "console=ttyS0" to the
qemu script, so we can scroll though the kernel log.
qemu-system-x86_64 -kernel <kernel-dir>/arch/x86_64/boot/bzImage \ -initrd <path-to>/ramdisk.img \ -m 512 -s -S \ -append "console=ttyS0"
qemu kernel instance remotely with
To be able to debug the Linux Kernel with
gdb we need the Linux Kernel symbols to be able to trace through the kernel Code, Lucky for us since we've compiled the Linux Kernel ourselves, if we navigate to the compiled
<kernel-dir> , we'd find a
vmlinux which is yet another Linux Kernel Image File, but this file is statically linked, containing all
debug_info and The Linux Kernel Symbols we need, So this is the Linux Kernel File we need to attach to
gdb to load the Kernel symbols.
Now that the Linux Kernel symbols are loaded in
gdb, and we have the Kernel
qemu instance waiting for the
gdb connection, we can connect with
target remote :1234 .
Now we are connected to the
qemu instance via its
GDBStub and can walk through the
main.c code and start exploring the Linux Kernel.
4→ Debugging the Linux Kernel:
Let's start by setting up a hardware breakpoint at
start_kernel() and hit continue to remotely control booting the kernel.
Now that doesn't seem like we control booting the kernel at all, because the kernel actually booted _ we entered the
initramfs recovery shell _ and our breakpoint was not hit.
That's because we need to disable the kernel ASLR by adding
nokaslr to the
qemu-system-x86_64 -kernel <kernel-dir>/arch/x86_64/boot/bzImage \ -initrd <path-to>/ramdisk.img \ -m 512 -s -S \ -append "console=ttyS0 nokaslr"
VOILA! now we actually control booting the kernel, from there you're free to walk through the code, learn the Linux Kernel by debugging, view and list the disassembly as well as the source code.
// Compiling the Linux Kernel might be a tough job, yet it comes with its pros as there is a directory in your
<kernel-dir> that has tons of helper scripts including
gdb scripts you can add to your
.gdbinit that's very much useful for debugging Kernel Modules.
you can view them in
gdb after adding the path to the script in
.gdbinit buy typing
lx- and hitting TAB .
# add path to the linux kernel gdb script for safe auto loading echo "add-auto-load-safe-path <kernel-dir>/vmlinux-gdb.py" >> ~/.gdbinit
And that's debugging the Linux Kernel with
qemu for you!, I hope you had fun going this far! If you found this setup interesting, maybe you can try out debugging the Linux Kernel with
virtualbox and the
:: Good Job and Later with another hack!! 👾👾