The memory system allows us to allocate the native memory that we need in which to store the machine code generated by our just-in-time compiler. We need to allocate our own native memory to store machine code because the processor will only execute code in memory with the executable permission, and higher level languages like Ruby don't give us anything to set memory permissions.
We implement our memory system using the system calls mmap, mprotect, and
munmap, which we call using the different foreign function interfaces of the
implementations of Ruby on which we run.
We need memory in which to store the machine code which we will generate. One
reason that we can't use normal Ruby objects, such as a String or Array for
this, is because Ruby's garbage collector can move objects around in memory
without telling us, and we need to be able to rely on being able to call our
machine code with a fixed address. We also can't use normal Ruby objects, or
even Ruby's underlying mechanism for allocating native memory, malloc, because
we need our memory to have the executable permission. Our processor won't
execute instructions from memory without this permission. It's the operating
system kernel that can set memory access permission, so we need memory that the
kernel is more managing. This is what the Rhizome memory system does - it asks
the kernel directly for native memory, it can set the read, write and execute
permissions, it can read and write the memory, and it can call the memory as a
machine code function.
The Rhizome memory system is therefore a low level tool of our just-in-time compiler, but a key one that unlocks the magic door that we need out of Ruby and into the world where we can run our own machine code.
The Rhizome memory system allocates memory using the mmap system call. A
system call is a library function provided by the operating system kernel. Like
many system calls, mmap does quite a lot. You may have heard of mmap in the
context of making files available as memory without having to manually read all
the data in. We use mmap in 'anonymous' mode, which means that we don't want
to load a file and we don't want to save any changes back to a file - we just
want the memory space.
A second system call mprotect allows us to change the access permissions for
memory that we've already allocated with mmap. We want to change the
permissions rather than just set them once when allocating, becuase we want to
have the memory writable but not executable while we write the code to it, and
then executable but not writable while we run it. This can increase safety, and
some operating systems require it.
The final system call that we use is munmap, which frees memory that we no
longer need. We do this manually as soon as we know we will no longer need the
memory, because Ruby's garbage collector doesn't know how much space it is
consuming and so may not know that it would be useful to free it when we become
short on space. We do use the Ruby garbage collector to free our native memory
if we forget to do it ourselves, such as if an exception is thrown and our code
to free it manually is not run.
Those are the system calls that we use. To make these system calls from Ruby
without using any gems we have to use a different mechanism for each
implementation of Ruby. On MRI we use the fiddle standard library. Rubinius
doesn't support fiddle, so there we use their built-in ffi
library. JRuby does support fiddle, but its version is not compatible with
MRI, so there we also have to use their built-in ffi library,
which isn't quite the same as the one in Rubinius. All of these libraries do the
same two things though - they let us call native functions from Ruby, and then
let us read and write native memory based on native addresses.
We could simplify our code here. If we used the ffi gem, rather than fiddle
or the versions of ffi built into Rubinius and JRuby, then the interface could
be standard for all implementations, but we have the self-imposed goal of
running on standard Ruby implementations with no additional code.
Calling mmap every time we want to allocate memory for a compiled method is
probably inefficient. I think it only works at the granularity of 'pages' of
memory as the kernel manages them, so even if we only need a few hundred bytes,
the kernel will round it up to a multiple of four thousand bytes or so. If we
wanted to run real Ruby applications this may add up to a great deal of wasted
space very quickly.
Most production just-in-time compilers will abstract from the kind of memory system we have here and will implement a 'code cache'. They will allocate the underlying memory that they need in the same way, but instead of doing it each time they compile a method, they will allocate a large chunk of memory and then fit compiled code into it as needed. The code cache will also have logic to remove compiled code when space becomes limited or the method is modified. This has all the usual complexities of caching - do you remove the largest compiled code? The least recently used? One at random?
We don't often think about just-in-time compilers as being a cache, but becuase you could recompile methods every time they were called if you wanted to, we can indeed think of storing the compiled version as just another cache.
- Implement a code cache that requests large blocks of memory and allocates from
them, rather than calling
mmapfor each allocation. - Implement code cache eviction when there is memory pressure.