Thursday, February 28, 2013

Adding a simple io device to gem5

Last time, I added custom pseudo-instructions in gem5. Today, I add a device in gem5 and then use the device from within a simulated (linux-x86_64) system.

Adding a device to gem5 is lightly documented in the ASPLOS tutorial and gem5 wiki. I would suggest starting with the tutorial, and read about the memory system as well.

Devices are located in gem5/src/dev/ subtree, with architecture-specific files located in subdirectories. The IsaFake device, which I found before the ASPLOS tutorial, was useful for starting. To create a simple device, I copied isa_fake.[cc|hh] to mydev.[cc|hh], and copied to Then I copied the parameters for IsaFake from into the parameters of, and added and to the SConscript. After renaming (search-replace IsaFake/BadDevice with MyDevice, isa_fake with mydev, etc), I needed to add the device to the system. I'm working with x86, so I attached it in the x86/ file, with:

from MyDevice import MyDevice
    my_device = MyDevice(pio_addr=x86IOAddress(0xe000), pio_size=8)
        self.fake_floppy.pio = bus.master
        self.my_device.pio = bus.master
        self.pciconfig.pio = bus.default

After compiling and running gem5 the device is listed in the m5out/config.ini file.

Accessing the device requires a device driver. To learn about writing drivers, read a good book. For this driver, a simple kernel module will do.
#include <linux kernel.h>
#include <linux module.h>
#include <linux errno.h>
#include <linux ioport.h>
#include <asm io.h>

#define BASE 0xe000
#define SIZE 0x08

int init_module(void)
  int t1;
  if ( ! request_region(BASE, SIZE, "mydev") ) {
    printk( KERN_INFO "unable to get io port at 0x%8X\n", BASE );
    return -ENODEV;

  /* a little test */
  t1 = inl(BASE);
  printk( KERN_INFO "read %d\n", t1 );

  outl(0, BASE);
  t1 = inl(BASE);
  printk( KERN_INFO "read %d\n", t1 );
  return 0;

void cleanup_module(void)
  release_region(BASE, SIZE);

Compile the module against the Linux kernel, boot gem5, get the module into the simulated system (e.g. with m5 readfile), and insert the module. With the default parameters from the IsaFake device, the write is ignored and the device returns -1 whenever it is read.

I did not get I/O memory working, but for now I/O ports are fine for me.

Tuesday, February 12, 2013

Add a pseudo instruction to gem5

An important aspect of many computer architecture projects is to modify an instruction set, often to extend the instructions with a new instruction that implements a proposed feature. I'm working on moving some of my research to the GEM5 open source simulator, but first I need to get an idea of the level of effort involved. My first move is to figure out how to add new instructions.

GEM5, being designed especially for computer architecture research, has a well-defined set of pseudo instructions that can be extended to serve my purposes. However, there are not really any instructions on how to extend these instructions. The few emails that I could find about pseudo instructions basically just said go look at what is implemented and extend it. So that is what I did. For posterity, I'll relay my findings here. Maybe they will be helpful to others, or to myself in the future.

The pseudo instructions are useful for implementing functional simulator features that can use a multiple-register instruction. The main drawback is that the pseudo instructions are not integrated tightly with the pipeline and are executed non-speculatively, so if the rate of your new instruction is quite high, the cost could be misleading if doing performance evaluations of the new feature. For my work, the pseudo instruction is fine; I have previously done a very similar implementation for functional simulation with Simics/GEMS.

Adding a new pseudo instruction (for X86)

I'm interested primarily in the X86 full-system simulation capabilities of GEM5 at the moment, so my effort is in that area. However, the pseudo instructions have implementations in the other architectures, and most of the following will translate directly to them.
  • Overwrite a reserved opcode in src/arch/x86/isa/decoder/two_byte_opcodes.isa near the other pseudo instructions (look for m5panic).
  • Add the instruction’s functional simulation implementation in src/sim/
  • Add the function prototype in src/sim/pseudo_inst.hh. The function prototype will define the available registers for parameters and return values based on the compiler’s calling conventions for the architecture.
  • Create an m5op for easily emitting the instruction in compiled code.
    • Add function number in util/m5/m5ops.h
    • Add function prototype in util/m5/m5op.h
    • Instantiate a TWO_BYTE_OP in m5op_x86.S
I have written a simple example that implements addition as a pseudo instruction. The patch may bit-rot, but the idea should be easy enough to follow.

To use the new pseudo instruction call the function declared in util/m5/m5op.h. Then (cross-)compile your source code with the m5 utilities like:
  gcc -o foo foo.c -I ${GEM5}/util/m5 ${GEM5}/util/m5/m5op_x86.S

To get your code into the simulation, you can
  • add the binary to the disk image 
    • sudo mount -o loop,offset=32256 /dist/m5/system/disks/linux-x86.img /mnt/tmp
    • cp foo /mnt/tmp/bin
  •  or read it directly into the simulation 
    • build/X86/gem5.debug configs/example/ -r 1 --script=foo
    • m5term localhost 3456
    • m5 readfile > foo
    • chmod +x foo
    • ./foo
Adding to the disk image requires restarting the simulation, whereas if you have a checkpoint loaded you can read the file in directly using m5 readfile.

Executing the pseudo instruction on real hardware

You can also use your new pseudo-instruction in real hardware by providing an illegal instruction handler (SIGILL handler) that emulates the functionality of the instruction. This may be useful for debugging purposes, since native hardware can run the emulation code much faster than the simulator will. I have written a simple example that shows how to handle the illegal instruction signal that gets caused when the pseudo instruction is executed. This sample example will execute in both GEM5 and natively (on a 64-bit X86).

I guess that covers it for now. Happy hacking!

Monday, February 11, 2013


I previously wrote about booting RTEMS on M5 (Now gem5) SPARC_FS. I am following up on this work in preparation for the release of RTEMS 4.11, which will hopefully be coming soon. I compiled RTEMS (with two small patches) and booted two samples on gem5 and on Simics Niagara.

On the simulator side, the instructions I gave previously continue to work for booting both OpenSolaris and RTEMS. I updated some of the links in the prior post to reflect the change in host to Oracle for the OpenSPARC tools. One issue I did not notice at first is that the port for the console to connect m5term to the gem5 simulator is port 3457.

I successfully built and booted hello and ticker with the niagara BSP. There are a few patches that need to be applied to RTEMS, but hopefully I can get those patches committed before releasing 4.11. Ideally, the RTEMS 4.11 will be able to compile and boot with gem5, giving the RTEMS community an open-source simulator for testing.

Interestingly, the ticker appears to count time. I'll have to take a look at how gem5 simulates the (s)tick register. Simics does not simulate the timer with any accuracy.

I think Qemu would be a nice target simulator for some future endeavors, but I do not have the time to investigate the feasibility of running RTEMS on the Qemu Sparc64.

Thursday, February 7, 2013


I finally learned how to use screen, a terminal windowing program that can detach so its windows run as a background task. When you detach, screen returns you to your shell. Detaching allows me to start work on a server from my lab workstation in a screen session, detach screen, logout and turn off my workstation, go home, and resume the same screen session from home.

Start screen with
$> screen
Once inside screen, the windows act mostly like a terminal, but now you can issue commands to screen using Ctrl-a and a command key. The useful keys that I have been using are:
  • Create a window (Ctrl-a c)
  • Next (Ctrl-a n) or Previous (Ctrl-a p) window.
  • Detach (Ctrl-a d)
To start screen with a previously detached session, first find the session with:
$> screen -ls 
There are screens on:
    11999.pts-0.localhost    (Attached)
    17129.pts-3.localhost    (Detached)
$> screen -r 17129.pts-3.localhost
The -r flag resumes the detached screen session.

Detaching is useful especially for long-running programs or for conserving the state of many terminal windows.