Adding a new System Call in Linux (x86 and x86_64)

। जय श्री भगवान् ।

In the previous module we saw how we can modify the running kernel by dynamically adding or removing modules from it. In this post I'll talk about how to add a new system to x86 or x86_64 system. Most of the stuff available on internet corresponds to 2.6 kernels however some files have been moved in order to make the changes required simple.

So let's dive into how we'll write a new system call. Firstly you will need the kernel sources and then you'll need to change the following file(s). I've tested this on a 32 bit system however 64 bit should be similar. 

Making changes to the sources 

 

1. Locate the directory, arch/x86/syscalls. This directory is having he syscall table files for both 32 and 64 bit.
2.  Open the file syscall_32.tbl for 32 bit and same way syscall_64.tbl for 64 bit.
3. You can see all the system calls listed here with their number on the extreme left. The format is also shown in the first line of the file. We won't discuss about which ABI to use but for 32 bit you can use i386.
4. At the end of this table (The last one listed would be #350 sys_finit_module). Now you'll need to add your system call here. Remember to set the correct number and the name. You won't need to supply a compat_ version of this. So you'll be having 3 entries like shown below

348     i386    process_vm_writev       sys_process_vm_writev           compat_sys_process_vm_writev
349     i386    kcmp                    sys_kcmp
350     i386    finit_module            sys_finit_module
351     i386    pks_first_call          sys_pks_first_call

So the above change shows that I added the system call named, pks_first_call however the entry point is going to be sys_pks_first_call. If you use the SYSCALL_DEFINE* macros then those macros would add sys_ to the name of the function hence the entry point is named that way.

Adding the system call code

To add the system call code, we'll create a new directory in the arch/x86 directory and then modify the Kbuild , top-level file to compile our new system call. So let's start by creating the required files first.

mkdir /usr/src/linux/arch/x86/pks_first

Now in this directory create a new file. I named the file same as my system call name however you can choose anything you want. So my file looks like as shown below,

#include <linux/kernel.h>
#include <linux/syscalls.h>

SYSCALL_DEFINE0(pks_first_call)
{
        printk (KERN_INFO "Inside %s",__FUNCTION__);
        return 0;
}

As you can see I've used SYSCALL_DEFINE0 since there's no argument for our system call. There are other versions of SYSCALL_DEFINE* so you are encouraged to use those since they also create ftrace meta data holder in addition in case CONFIG_FTRACE is enabled.

In the same directory you'll also need to have a Makefile which does nothing but tells which files need to be compiled so it's very simple as shown below,

obj-y           += pks_first_call.o

Changing the Top-Level Kbuild

All we need to do now is change the top level KBuild file. This would be the one located in arch/x86/Kbuild. The listing is as shown below,

ifneq ($(CONFIG_XEN),y)
obj-y += realmode/
endif
obj-y += kernel/
obj-y += mm/

obj-y += crypto/
obj-y += vdso/
obj-$(CONFIG_IA32_EMULATION) += ia32/

obj-y += platform/
obj-y += net/
obj-y += pks_first/

I added my directory in the top level KBuild at the end. The final change we need to do is let the kernel know about the system call. So we need to change the following file to finish all our changes
include/linux/syscalls.h. The change looks like as shown below,

asmlinkage long sys_kcmp(pid_t pid1, pid_t pid2, int type,
                         unsigned long idx1, unsigned long idx2);
asmlinkage long sys_finit_module(int fd, const char __user *uargs, int flags);
asmlinkage long sys_pks_first_call(void);
#endif  

In the above listing I've added my system call at the end right before #endif. We don't need to change the __NR_syscalls it'll be taken care of by the build system for x86. You can build and install your new kernel and check your new system call with a simple program as shown below.

#include <stdio.h>
#include <syscall.h>
#include <errno.h>
#define NR_pks_first_call 351
int main()
{
        if (syscall(NR_pks_first_call)) {
                perror("OOPS:");
        } 
        return 0;
}

When you run the above program you shouldn't get any message but if you look in dmesg output or your system log file usually /var/log/messages you should be able to see the message posted by our system call. This was quite a lot of information and next time we'll see other ways we can interact with kernel using something simpler instead of recompiling and installing the whole kernel.