Writing a Linux Kernel Module — Part 1: Introduction

Introduction

In this series of articles I describe how you can write a Linux kernel module for an embedded Linux device. I begin with a straightforward “Hello World!” loadable kernel module (LKM) and work towards developing a module that can control GPIOs on an embedded Linux device (such as the BeagleBone) through the use of IRQs. I will add further follow-up articles as I identify suitable applications.

This is a complex topic that will take time to work through. Therefore, I have broken the discussion up over a number of articles, each providing a practical example and outcome. There are entire books written on this topic, so it will be difficult to cover absolutely every aspect. There are also other articles available on writing kernel modules; however, the examples presented here are built and tested under the Linux kernel 3.8.X+, ensuring that the material is up to date and relevant, and I have focused on interfacing to hardware on embedded systems. I have also aligned the tasks performed against my book, Exploring BeagleBone, albeit the articles are self-contained and do not require that you own a copy of the book.

This article is focused on the system configuration, tools and code required to build and deploy a “Hello World!” kernel module. The second article in this series examines the topic of writing character device drivers and how to write C/C++ programs in user space that can communicate with kernel space modules. The third article examines the use of the kernel space GPIO library code — it combines the content of the first two articles to develop interrupt-driven code that can be controlled from Linux user space. For example, Figure 1 illustrates an oscilloscope capture of an interrupt-driven kernel module that triggers an LED to light when a button is pressed (click for a larger version). Under regular embedded Linux (i.e., not a real-time variant), this code demonstrates a response time of approximately 20 microseconds (±5μs), with negligible CPU overhead.

What is a Kernel Module?

A loadable kernel module (LKM) is a mechanism for adding code to, or removing code from, the Linux kernel at run time. They are ideal for device drivers, enabling the kernel to communicate with the hardware without it having to know how the hardware works. The alternative to LKMs would be to build the code for each and every driver into the Linux kernel.

Without this modular capability, the Linux kernel would be very large, as it would have to support every driver that would ever be needed on the BBB. You would also have to rebuild the kernel every time you wanted to add new hardware or update a device driver. The downside of LKMs is that driver files have to be maintained for each device. LKMs are loaded at run time, but they do not execute in user space — they are essentially part of the kernel.

Kernel modules run in kernel space and applications run in user space, as illustrated in Figure 2. Both kernel space and user space have their own unique memory address spaces that do not overlap. This approach ensures that applications running in user space have a consistent view of the hardware, regardless of the hardware platform. The kernel services are then made available to the user space in a controlled way through the use of system calls. The kernel also prevents individual user-space applications from conflicting with each other or from accessing restricted resources through the use of protection levels (e.g., superuser versus regular user permissions).

Why Write a Kernel Module?

When interfacing to electronics circuits under embedded Linux you are exposed to sysfs and the use of low-level file operations for interfacing to electronics circuits. This approach can appear to be inefficient (especially if you have experience of traditional embedded systems); however, these file entries are memory mapped and the performance is sufficient for many applications. I have demonstrated in my book that it is possible to achieve response times of about one third of a millisecond, with negligible CPU overhead, from within Linux user space by using pthreads, callback functions and sys/poll.h.

An alternative approach is to use kernel code, which has support for interrupts. However, kernel code is difficult to write and debug. My advice is that you should always to try to accomplish your task in Linux user space, unless you are certain that there is no other possible way!

[tagline_box backgroundcolor=”” shadow=”yes” shadowopacity=”0.7″ border=”0px” bordercolor=”” highlightposition=”top” content_alignment=”left” link=”https://github.com/derekmolloy/exploringBB/tree/master/extras/kernel/” linktarget=”_blank” modal=”” button_size=”” button_shape=”” button_type=”” buttoncolor=”” button=”Get Source Code” title=”Source Code for this Discussion” description=”” animation_type=”0″ animation_direction=”down” animation_speed=”0.1″ class=”” id=””]

All of the code for this discussion is available in the GitHub repository for the book Exploring BeagleBone. The code can be viewed publicly at: the ExploringBB GitHub Kernel Project directory, and/or you can clone the repository on your BeagleBone (or other Linux device) by typing:

The /extras/kernel/hello directory is the most important resource for this article. The auto-generated Doxygen documentation for these code examples is available in HTML format and PDF format.

[/tagline_box]

Prepare the System for Building LKMs

The system must be prepared to build kernel code, and to do this you must have the Linux headers installed on your device. On a typical Linux desktop machine you can use your package manager to locate the correct package to install. For example, under 64-bit Debian you can use:
molloyd@DebianJessieVM:~$ sudo apt-get update
molloyd@DebianJessieVM:~$ apt-cache search linux-headers-$(uname -r)
linux-headers-3.16.0-4-amd64 - Header files for Linux 3.16.0-4-amd64
molloyd@DebianJessieVM:~$ sudo apt-get install linux-headers-3.16.0-4-amd64
molloyd@DebianJessieVM:~$ cd /usr/src/linux-headers-3.16.0-4-amd64/
molloyd@DebianJessieVM:/usr/src/linux-headers-3.16.0-4-amd64$ ls
arch include Makefile Module.symvers scripts

You can complete the first two articles in this series using any flavor of desktop Linux. However, in this series of articles I build the LKM on the BeagleBone itself, which simplifies the process when compared to cross-compiling. You must install the headers for the exact version of your kernel build. Similar to the desktop installation, use uname to identify the correct installation. For example:
molloyd@beaglebone:~$ uname -a
Linux beaglebone 3.8.13-bone70 #1 SMP Fri Jan 23 02:15:42 UTC 2015 armv7l GNU/Linux

You can download the Linux headers for the BeagleBone platform from Robert Nelson’s website. For example, at: http://rcn-ee.net/deb/precise-armhf/. Choose the exact kernel build, and download and install those Linux-headers on your BeagleBone. For example:
molloyd@beaglebone:~/tmp$ wget http://rcn-ee.net/deb/precise-armhf/v3.8.13-bone70/linux-headers-3.8.13-bo
ne70_1precise_armhf.deb
100%[===========================>] 8,451,080 2.52M/s in 3.2s
2015-03-17 22:35:45 (2.52 MB/s) - 'linux-headers-3.8.13-bone70_1precise_armhf.deb' saved [8451080/8451080]
molloyd@beaglebone:~/tmp$ sudo dpkg -i ./linux-headers-3.8.13-bone70_1precise_armhf.deb
Selecting previously unselected package linux-headers-3.8.13-bone70

You can then check that the headers have installed correctly:
molloyd@beaglebone:~/tmp$ cd /usr/src/linux-headers-3.8.13-bone70/
molloyd@beaglebone:/usr/src/linux-headers-3.8.13-bone70$ ls
Documentation Module.symvers crypto fs ipc mm scripts tools
Kconfig arch drivers include kernel net security usr
Makefile block firmware init lib samples sound virt

Under the 3.8.13-bone47 Debian distribution for the BeagleBone, you may have to perform an unusual step of creating an empty file timex.h (i.e., touch timex.h) in the directory /usr/src/linux-headers-3.8.13-bone47/arch/arm/
include/mach. This step is not necessary under the bone70 build.

[tagline_box backgroundcolor=”” shadow=”yes” shadowopacity=”0.7″ border=”0px” bordercolor=”” highlightposition=”top” content_alignment=”left” link=”” linktarget=”” modal=”” button_size=”” button_shape=”” button_type=”” buttoncolor=”” button=”” title=”A Warning!” description=”” animation_type=”0″ animation_direction=”down” animation_speed=”0.1″ class=”” id=””]

It is very easy to crash the system when you are writing and testing LKMs. It is always possible that such a system crash could corrupt your file system — it is unlikely, but it is possible. Please back up your data and/or use an embedded system, such as the BeagleBone, which can easily be re-flashed. Performing a sudo reboot, or pressing the reset button on the BeagleBone will usually put everything back in order. No BeagleBones were corrupted in the writing of these articles despite many, many system crashes!

[/tagline_box]

The Module Code

The run-time life cycle of a typical computer program is reasonably straightforward. A loader allocates memory for the program, then loads the program and any required shared libraries. Instruction execution begins at some entry point (typically the main() point in C/C++ programs), statements are executed, exceptions are thrown, dynamic memory is allocated and deallocated, and the program eventually runs to completion. On program exit, the operating system identifies any memory leaks and frees lost memory to the pool.

A kernel module is not an application — for a start there is no main() function! Some of the key differences are that kernel modules:

• do not execute sequentially— a kernel module registers itself to handle requests using its initialization function, which runs and then terminates. The type of requests that it can handle are defined within the module code. This is quite similar to the event-driven programming model that is commonly utilized in graphical-user interface (GUI) applications.
• do not have automatic cleanup — any resources that are allocated to the module must be manually released when the module is unloaded, or they may be unavailable until a system reboots.
• do not have printf() functions — kernel code cannot access libraries of code that is written for the Linux user space. The kernel module lives and runs in kernel space, which has its own memory address space. The interface between kernel space and user space is clearly defined and controlled. We do however have a printk() function that can output information, which can be viewed from within user space.
• can be interrupted — one conceptually difficult aspect of kernel modules is that they can be used by several different programs/processes at the same time. We have to carefully construct our modules so that they have a consistent and valid behavior when they are interrupted. The BeagleBone has a single-core processor (for the moment) but we still have to consider the impact of multiple processes accessing the module simultaneously.
• have a higher level of execution privilege — typically, more CPU cycles are allocated to kernel modules than to user-space programs. This sounds like an advantage, however, you have to be very careful that your module does not adversely affect the overall performance of your system.
• do not have floating-point support — it is kernel code that uses traps to transition from integer to floating-point mode for your user space applications. However, it is very difficult to perform these traps in kernel space. The alternative is to manually save and restore floating point operations — a task that is best avoided and left to your user-space code.

The concepts above are a lot to digest and it is important that they are all addressed, but not all in the first article! Listing 1 provides the code for a first example LKM. When no kernel argument is provided, the code uses the printk() function to display “Hello world!…” in the kernel logs. If the argument “Derek” is provided, then the logs will display “Hello Derek!…” The comments in Listing 1, which are written using a Doxygen style, describe the role of each statement. Further description is available after the code listing below.

In addition to the points described by the comments in Listing 1, there are some additional points:

• Line 16: The statement MODULE_LICENSE("GPL") provides information (via modinfo) about the licensing terms of the module that you have developed, thus allowing users of your LKM to ensure that they are using free software. Since the kernel is released under the GPL, your license choice impacts upon the way that the kernel treats your module. You can choose "Proprietary" for non-GPL code, but the kernel will be marked as “tainted” and a warning will appear. There are non-tainted alternatives to GPL, such as "GPL v2", "GPL and additional rights", "Dual BSD/GPL", "Dual MIT/GPL", and "Dual MPL/GPL". See linux/module.h for more information.
• Line 21: The name (ptr to char) is declared as static and is initialized to contain the string “hello”. You should avoid using global variables in kernel modules — it is even more important than in application programming, as global variables are shared kernel wide. You should use the static keyword to restrict a variable’s scope to within the module. If you must use a global variable, add a prefix that is unique to the module that you are writing.
• Line 22: The module_param(name, type, permissions) macro has three parameters: name (the parameter name displayed to the user and the variable name in the module), type (the type of the parameter — i.e., one of byte, int, uint, long, ulong, short, ushort, bool, an inverse Boolean invbool, or a char pointer charp), and permissions (this is the access permissions to the the parameter when using sysfs and is covered below. A value of 0 disables the entry, but S_IRUGO allows read access for user/group/others — See the Mode Bits for Access Permissions Guide)
• Line 31 and 40: The functions can have whatever names you like (e.g., helloBBB_init() and helloBBB_exit()), however, the same names must be passed to the special macros module_init() and module_exit() on lines 48 and 49.
• Line 31: The printk() is very similar in usage to the printf() function that you should be familiar with, and you can call it from anywhere within the kernel module code. The only significant difference is that you should specify a log level when you call the function. The log levels are defined in linux/kern_levels.h as one of KERN_EMERG, KERN_ALERT, KERN_CRIT, KERN_ERR, KERN_WARNING, KERN_NOTICE, KERN_INFO, KERN_DEBUG, and KERN_DEFAULT. This header is included via the linux/kernel.h header file, which includes it via linux/printk.h.

Essentially, when this module is loaded the helloBBB_init() function will execute, and when the module is unloaded the helloBBB_exit() function will execute.

The next step is to build this code into a kernel module.

Building the Module Code

A Makefile is required to build the kernel module — in fact, it is a special kbuild Makefile. The kbuild Makefile required to build the kernel module in this article can be viewed in Listing 2.

The first line of this Makefile is called a goal definition and it defines the module to be built (hello.o). The syntax is surprisingly intricate, for example obj-m defines a loadable module goal, whereas obj-y indicates a built-in object goal. The syntax becomes more complex when a module is to be built from multiple objects, but this is sufficient to build this example LKM.

The reminder of the Makefile is similar to a regular Makefile. The $(shell uname -r) is a useful call to return the current kernel build version — this ensures a degree of portability for the Makefile. The -C option switches the directory to the kernel directory before performing any make tasks. The M=$(PWD) variable assignment tells the make command where the actual project files exist. The modules target is the default target for external kernel modules. An alternative target is modules_install which would install the module (the make command would have to be executed with superuser permissions and the module installation path is required).

All going well, the process to build the kernel module should be straightforward, provided that you have installed the Linux headers as described earlier. The steps are as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ ls -l
total 8
-rw-r--r-- 1 molloyd molloyd 154 Mar 17 17:47 Makefile
-rw-r--r-- 1 molloyd molloyd 2288 Apr 4 23:26 hello.c
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ make
make -C /lib/modules/3.8.13-bone70/build/ M=/home/molloyd/exploringBB/extras/kernel/hello modules
make[1]: Entering directory '/usr/src/linux-headers-3.8.13-bone70'
CC [M] /home/molloyd/exploringBB/extras/kernel/hello/hello.o
Building modules, stage 2.
MODPOST 1 modules
CC /home/molloyd/exploringBB/extras/kernel/hello/hello.mod.o
LD [M] /home/molloyd/exploringBB/extras/kernel/hello/hello.ko
make[1]: Leaving directory '/usr/src/linux-headers-3.8.13-bone70'
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ ls
Makefile Module.symvers hello.c hello.ko hello.mod.c hello.mod.o hello.o modules.order

You can see that there is now a hello loadable kernel module in the build directory with the file extension .ko.

Testing the LKM

This module can now be loaded using the kernel module tools as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ ls -l *.ko
-rw-r--r-- 1 molloyd molloyd 4219 Apr 4 23:27 hello.ko
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ sudo insmod hello.ko
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ lsmod
Module Size Used by
hello 972 0
g_multi 50407 2
libcomposite 15028 1 g_multi
omap_rng 4062 0
mt7601Usta 639170 0

You can get information about the module using the modinfo command, which will identify the description, author and any module parameters that are defined:
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ modinfo hello.ko
filename: /home/molloyd/exploringBB/extras/kernel/hello/hello.ko
description: A simple Linux driver for the BBB.
author: Derek Molloy
license: GPL
srcversion: 9E3F5ECAB0272E3314BEF96
depends:
vermagic: 3.8.13-bone70 SMP mod_unload modversions ARMv7 thumb2 p2v8
parm: name:The name to display in /var/log/kernel.log. (charp)

The module can be unloaded using the rmmod command:
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ sudo rmmod hello.ko

You can repeat these steps and view the output in the kernel log that results from the use of the printk() function. I recommend that you use a second terminal window and view the output as your LKM is loaded and unloaded, as follows:
molloyd@beaglebone:~$ sudo su -
[sudo] password for molloyd:
root@beaglebone:~# cd /var/log
root@beaglebone:/var/log# tail -f kern.log
...
Apr 4 23:34:32 beaglebone kernel: [21613.495523] EBB: Hello world from the BBB LKM!
Apr 4 23:35:17 beaglebone kernel: [21658.306647] EBB: Goodbye world from the BBB LKM!
^C
root@beaglebone:/var/log#

Testing the LKM Custom Parameter

The code in Listing 1 also contains a custom parameter, which allows an argument to be passed to the kernel module on initialization. This feature can be tested as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ sudo insmod hello.ko name=Derek

If you view /var/log/kern.log at this point then you will see “Hello Derek” in place of “Hello world”. However, it is worth having a look at /proc and /sys first.

Rather than using the lsmod command, you can also find out information about the kernel module that is loaded, as follows:
molloyd@beaglebone:~/exploringBB/extras/kernel/hello$ cd /proc
molloyd@beaglebone:/proc$ cat modules|grep hello
hello 972 0 - Live 0xbf903000 (O)
This is the same information that is provided by the lsmod command but it also provides the current kernel memory offset for the loaded module, which is useful for debugging.

The LKM also has an entry under /sys/module, which provides you with direct access to the custom parameter state. For example:
molloyd@beaglebone:/proc$ cd /sys/module
molloyd@beaglebone:/sys/module$ ls -l|grep hello
drwxr-xr-x 6 root root 0 Apr 5 00:02 hello
molloyd@beaglebone:/sys/module$ cd hello
molloyd@beaglebone:/sys/module/hello$ ls -l
total 0
-r--r--r-- 1 root root 4096 Apr 5 00:03 coresize
drwxr-xr-x 2 root root 0 Apr 5 00:03 holders
-r--r--r-- 1 root root 4096 Apr 5 00:03 initsize
-r--r--r-- 1 root root 4096 Apr 5 00:03 initstate
drwxr-xr-x 2 root root 0 Apr 5 00:03 notes
drwxr-xr-x 2 root root 0 Apr 5 00:03 parameters
-r--r--r-- 1 root root 4096 Apr 5 00:03 refcnt
drwxr-xr-x 2 root root 0 Apr 5 00:03 sections
-r--r--r-- 1 root root 4096 Apr 5 00:03 srcversion
-r--r--r-- 1 root root 4096 Apr 5 00:03 taint
--w------- 1 root root 4096 Apr 5 00:02 uevent
-r--r--r-- 1 root root 4096 Apr 5 00:02 version
molloyd@beaglebone:/sys/module/hello$ cat version
0.1
molloyd@beaglebone:/sys/module/hello$ cat taint
O
The version value is 0.1 as per the MODULE_VERSION("0.1") entry and the taint value is 0 as per the license that has been chosen, which is MODULE_LICENSE("GPL").

The custom parameter can be viewed as follows:
molloyd@beaglebone:/sys/module/hello$ cd parameters/
molloyd@beaglebone:/sys/module/hello/parameters$ ls -l
total 0
-r--r--r-- 1 root root 4096 Apr 5 00:03 name
molloyd@beaglebone:/sys/module/hello/parameters$ cat name
Derek
You can see that the state of the name variable is displayed, and that superuser permissions where not required to read the value. The latter is due to the S_IRUGO argument that was used in defining the module parameter. It is possible to configure this value for write access but your module code will need to detect such a state change and act accordingly. Finally, you can remove the module and observe the output:
molloyd@beaglebone:/sys/module/hello/parameters$ sudo rmmod hello.ko

As expected, this will result in the output message in the kernel logs:
root@beaglebone:/var/log# tail -f kern.log
…
Apr 5 00:02:20 beaglebone kernel: [23281.070193] EBB: Hello Derek from the BBB LKM!
Apr 5 00:08:18 beaglebone kernel: [23639.160009] EBB: Goodbye Derek from the BBB LKM!

Conclusions

Click for the HTML and PDF version of the auto-generated Doxygen code documentation

Hopefully you have built your first loadable kernel module (LKM). Despite the simplicity of the functionality of this module there was a lot of material to cover — by the end of this article: you should have a broad idea of how loadable kernel modules work; you should have your system configured to build, load and unload such modules; and, you should be able to define custom parameters for your LKMs.

The next step is to build on this work to develop a kernel space LKM that can communicate with a user space C/C++ program by developing a basic character driver. See “Writing a Linux Kernel Module — Part 2: A Character Device“. Then we can move on to the more interesting task of interacting with GPIOs.