mmaping guarded pages

in one of my forths i use a guarded page for the working stack. this guarantees a segfault if memory on either direct side of the stack (which is just a single memory page) is accessed. here's the slightly nifty way in which i did this.

as it is a forth it is written in x64 assembly, but i will include the C library equivalent as well.

mov rsi, MEM_SIZE

add rsi, PAGE_SIZE * 2

MEM_SIZE is the size of the amount of memory we need. mmap allocates in memory pages, so it should be a multiple of the page size (on x64 non-big pages can only be 4096 bytes, so luckily you don't have to do some horrible bullshit to figure out the page size on that platform)

alongside our desired allocation size we need two extra pages, these will be the Guards and interacting with them will cause a segfault.

we do this maths in rsi to set up for the mmap system call's length parameter which is passed through rsi.

the rest of the parameters to mmap are constants:

mov r9, 0

mov r8, -1

r9 and r8 are the offset and fd parameters respectively. for our purposes they are entirely irrelevant because we are not mapping a file.

mov r10, 0x22

this is the flags parameter, it must contain one of three values and can have a few other values orred onto it. 0x22 is MAP_PRIVATE (0x02) | MAP_ANONYMOUS (0x20).

MAP_PRIVATE does basically what it says, it makes the mapping private to the process. the alternatives MAP_SHARED and MAP_SHARED_VALIDATE seem to only make any sense for mapping files, which we are not doing.

MAP_ANONYMOUS indicates that we are not mapping a file and want a zero initialised page of memory.

mov rdx, 0

this sets the prot parameter to PROT_NONE, marking the memory area as having no read/write/execute permissions. accessing the memory at all will cause a segfault.

'but doesn't that include the part we actually want to access too?' yes. we will fix this afterwards.

mov rdi, 0

mov rax, 9

syscall

rdi is the addr parameter, which if set to zero, which it is here, will just use whatever address is available. you could use this to tell mmap where to try to allocate the memory, if you wanted to.

and in rax we put the syscall for mmap which is 9, and everything is complete. here's all that in C:

mmap(0, MEM_SIZE + PAGE_SIZE * 2, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

the result of the syscall is returned in rax. it will either be a memory address or a negative number corresponding to -errno. we can filter out a failure with

test rax, rax

jl .fail

now we have a bit of memory which we cannot access. let's fix that. the strategy here is to now change thhe permissions of MEM_SIZE leaving a page before and after the allocation unaffected and still inaccessible.

mov r15, rax

add r15, PAGE_SIZE

we'll keep the start of our block of memory in r15. it will still be there after the syscall so we can use it for Storing Stuff.

mov rdx, 3 ; prot

mov rsi, MEM_SIZE ; size

mov rdi, r15 ; addr

mov rax, 10

syscall

10 is the syscall number for mprotect which lets us change the permissions on a block of memory. rsi (size) and rdi (addr) dictate, Shockingly, the size and address of the memory we want to change.

the prot value here is PROT_READ (0x1) | PROT_WRITE (0x2) which sets read and write permissions on the memory space.

congrations. we now have a block of memory with one page on either side that will safely(?) segfault when your program Fucks up.

here's that in C. goodbye + sorry about how im bad at writing and explaining things, i hope this helped anyway (a is the address here)

mprotect(a, MEM_SIZE, PROT_READ | PROT_WRITE);

thank you