Introduction
I wrote a cool reversing challenge (and one comparatively lamer one) for this year's SCTF qualifiers. I am predicting no one will solve it, so here is a compilation of all the cool obfuscation tricks I developed in service of it. They will be roughly ordered from least consequential to most consequential. I am not fully sure if any of these are 'novel' (I cannot imagine they are), but I came up with them largely independently and thought they were neat. I hope you will find them neat as well.
Note: 11 teams ended up solving it, and I am convinced none of them were human. I did have one human playtester though, thank you muong ^_^
First trick - Length check
As with all flag checkers, we want a length check (in my case, 50 chars incl newline). A lot of the challenge is implemented with syscalls, so naturally, we read the input from stdin with the read syscall. The read syscall returns the number of bytes read in rax, so I installed a SIGSEGV handler on hlt and use the returned rax value (the length of the provided input) as an indirect jmp.
404a27: 48 31 c0 xor rax,rax
404a2a: e8 f4 e6 ff ff call 403123 <memset@plt+0x20b3>
404a2f: 48 8d 14 25 3c 4a 40 lea rdx,ds:0x404a3c
404a36: 00
404a37: 48 01 c2 add rdx,rax
404a3a: ff e2 jmp rdx
404a3c: f4 hlt
404a3d: f4 hlt
404a3e: f4 hlt
404a3f: f4 hlt
404a40: f4 hlt
...The call to 0x403123 is a syscall (more on this trick later), so all this block of code does is take the number of bytes read and jump forward that many bytes in the binary. As we can see, there is a gigantic, bountiful sea of hlt instructions that would cause the binary to segfault and prematurely exit. However, exactly 50 bytes away from this jump instruction...
404a6a: f4 hlt
404a6b: f4 hlt
404a6c: f4 hlt
404a6d: f4 hlt
404a6e: e9 00 01 00 00 jmp 404b73 <umask@@Base+0x1838>
404a73: f4 hltthere is a precisely positioned jmp instruction that allows us to continue execution. I find the visual aspect (the Dirac Sea of HLTs) to be quite pleasing, so I like this trick.
Second trick - Patching the GOT + trampoline
This trick is largely inconsequential and does not need to be reversed. Note in the above trick, we call 0x403123.
404a27: 48 31 c0 xor rax,rax
404a2a: e8 f4 e6 ff ff call 403123 <memset@plt+0x20b3>
404a2f: 48 8d 14 25 3c 4a 40 lea rdx,ds:0x404a3cWhat does 0x403123 actually do? This is a small dispatcher routine that loads 0x401050 into rcx and then jmps to it.
*-> 0x403123 488d0c2550104000 <NO_SYMBOL> lea rcx, ds:0x401050
0x40312b 4839d9 <NO_SYMBOL> cmp rcx, rbx
0x40312e 7403 <NO_SYMBOL> je 0x403133
0x403130 7501 <NO_SYMBOL> jne 0x4031330x401050, is, of course, in the Global Offset Table. I hope if you are a reader of my blog you already know what the GOT is. This address, specifically, is the sigaction entry.
Name | PLT | GOT | GOT value
----- .rela.dyn -----
__libc_start_main | Not found | 0x00000040fe40 | 0x7ffff7dd7fa0 <__libc_start_main>
__gmon_start__ | Not found | 0x00000040fe48 | 0x000000000000
----- .rela.plt -----
_exit | 0x000000401030 | 0x00000040fe68 | 0x000000401036 <.plt+0x16>
puts | 0x000000401040 | 0x00000040fe70 | 0x000000401046 <.plt+0x26>
sigaction | 0x000000401050 | 0x00000040fe78 | 0x7ffff7e41022 <__lll_lock_wake_private+0x12>
printf | 0x000000401060 | 0x00000040fe80 | 0x000000401066 <.plt+0x46>
memset | 0x000000401070 | 0x00000040fe88 | 0x7ffff7f17d40
gef> sigaction is already resolved, but we can see that it is pointing to the wrong address (it is pointing to __lll_lock_wake_private) instead of the proper sigaction function. This is because it is being patched at runtime after the necessary sighandlers are already installed with sigaction. The disassembly just happens to point to this syscall ; ret gadget.
gef> x/10i 0x7ffff7e41022
0x7ffff7e41022 <__lll_lock_wake_private+18>: syscall
0x7ffff7e41024 <__lll_lock_wake_private+20>: ret
0x7ffff7e41025: data16 cs nop WORD PTR [rax+rax*1+0x0]
0x7ffff7e41030: xor sil,0x81
0x7ffff7e41034: xor r10d,r10d
0x7ffff7e41037: mov edx,0x1
0x7ffff7e4103c: mov eax,0xca
0x7ffff7e41041: syscall
0x7ffff7e41043: ret
0x7ffff7e41044: cs nop WORD PTR [rax+rax*1+0x0]At runtime, the binary gets the address of the sigaction PLT entry, reads it, and then walks libc for a syscall ; ret gadget from that address onwards. Afterwards, it overwrites the GOT to point to our located address.
1 void fla(void) {
2 │ // find libc address
3 │ __asm__(
4 │ │ "lea rax, sigaction\n"
5 │ │ "add rax, 0x2\n"
6 │ │ "mov ebx, dword ptr [rax]\n"
7 │ │ "add rbx, rax\n"
8 │ │ "push qword ptr [rbx + 0x4]\n"
9 │ │ "pop r14\n"
10 │ );
11 │
12 │ // overwrite got
13 │ __asm__(
14 │ │ "push r14\n"
15 │ │ "pop r15\n"
16 │ │ "find_syscall:\n"
17 │ │ "mov ax, word ptr [r15]\n"
18 │ │ "cmp ax, 0x050f\n"
19 │ │ "je fsdone\n"
20 │ │ "nope:\n"
21 │ │ "inc r15\n"
22 │ │ "jmp find_syscall\n"
23 │ │ "fsdone:\n"
24 │ │ "cmp byte ptr [r15 + 2], 0xc3\n"
25 │ │ "jne nope\n"
26 │ │ "mov qword ptr [rbx + 0x4], r15\n"
27 │ );
28 }Once again, the code is all handwritten in ASM, because I did not want any part of this binary to be decompilable (for this function though, I failed a bit). There are a handful of very minute attempts at code obfuscation (you can see a push and a pop instead of a direct mov instruction, for example).
This is directly inspired by the xzutils backdoor and its embedded x86 disassembler, more on this insanely beautiful technique here. The technique I implemented is far, far simpler, but the manual scanning is still necessary in order to make this dynamically linked binary libc agnostic. My motivation for this is as follows: wouldn't it be really fucking funny if the SROP challenge didn't actually have a syscall instruction inside it? I think a lot more can be done with this technique, but it really is just a tiny little curiosity in the initialisation code (which any competent human reverse engineer would skip over anyway).
An entire challenge could be based off of patching GOT entries at runtime. Hm.. that's an idea...
Third trick - abusing function alignment
The binary has around 10 of these functions:
28 __attribute__((aligned(0x10)))
27 void set_nibble_9() {
26 │ __asm__(
25 │ │ "call ret\n"
24 │ │ "movzx ebx, byte ptr [rsp - 0x8]\n"
23 │ │ "and bl, 0xf\n"
22 │ );
21 }
20
19 __attribute__((aligned(0x10)))
18 void set_nibble_a() {
17 │ __asm__(
16 │ │ "clc\n"
15 │ │ "call ret\n"
14 │ │ "movzx ebx, byte ptr [rsp - 0x8]\n"
13 │ │ "and bl, 0xf\n"
12 │ );
11 }
10
9 __attribute__((aligned(0x10)))
8 void set_nibble_b() {
7 │ __asm__(
6 │ │ "inc ebx\n"
5 │ │ "call ret\n"
4 │ │ "movzx ebx, byte ptr [rsp - 0x8]\n"
3 │ │ "and bl, 0xf\n"
2 │ );
1 }They all call an address with a lone ret instruction, and then pull the saved return address from the spent stack frame by scanning directly above it on the stack. These functions are all identical, except that they have different instructions preceding the call that are all inconsequential: call, inc ebx (when ebx is directly overwritten), inc rbx.
The goal of these 'padding' instructions is that they are functionally differently-sized NOPs. When our functions are compiled and the function prologues and epilogues are added, the call instruction is precisely positioned such that the address of the instruction succeeding it (i.e., our return address, which is pushed onto the stack and then later isolated) has a different nibble at the end.
40321f: 00
403220: 55 push rbp
403221: 48 89 e5 mov rbp,rsp
403224: bb ef be ad de mov ebx,0xdeadbeef
403229: e8 f4 fe ff ff call 403122 <memset@plt+0x20b2>
40322e: 0f b6 5c 24 f8 movzx ebx,BYTE PTR [rsp-0x8]
403233: 80 e3 0f and bl,0xf
403236: 90 nop
403237: 5d pop rbp
403238: c3 ret
403239: 0f 1f 80 00 00 00 00 nop DWORD PTR [rax+0x0]Note the illustrative example above. When the call instruction is triggered, the return address 0x40322e is pushed onto the stack. The program immediately returns, and [rsp - 0x08] now points to 0x40322e, and 0x0e is isolated and stored into rbx. Note that this only happens because our padding instruction, mov ebx, 0xdeadbeef, is 5 bytes long. If it were 4, 3, or 2 bytes, then the isolated nibble is always different, as the stored return address would shift accordingly.
Fourth trick - fake functions and garbage bytes
The flag-checking functions are all easily stereotypeable, as they follow a very predictable pattern of instructions. To guard against a solver simply scanning for function bytes in the binary and dumping all the constraints, I inserted a bunch of fake functions that follow the exact same pattern and inserted them throughout the binary randomly, interleaving them between the real flag-checker functions and the fake ones. This is a relatively standard trick that defeats naive static analysis; the way to isolate the actual functions is to just perform a trace and see what functions fire and which ones don't. There is not much else to say about this trick.
Additionally, what seemed to trip up a lot of agents is this fake function I inserted:
4 __attribute__((aligned(0x1000)))
3 __attribute__((naked))
2 volatile void base(void) {
1 │ printf("That's the correct flag!");
485 }This function never fires. To prevent gcc from optimising it out, we specify that it is a volatile function (hopefully you already know this). Since the string is unobfuscated and clearly exists in the binary within a function, the agent spends a lot of time trying to find where this function might be called. In a binary with significant control flow obfuscation, this quickly leads to a dead end.
Finally, junk instructions through the incbin directive (my favorite assembler directive (because people have those, of course)).
15 __attribute__((naked))
14 volatile void meow(void) {
13 │ __asm__(
12 │ │ ".incbin \"junk\"\n"
11 │ │ "ret:\nret\n"
10 │ │ "sc:\n"
9 │ │ "lea rcx, sigaction\n"
8 │ │ "cmp rcx, rbx\n"
7 │ │ "je ster\n"
6 │ │ "jne ster\n"
5 │ │ ".byte 0xe8\n"
4 │ │ "ster: jmp rcx\n"
3 │ │ ".incbin \"junk2\"\n"
2 │ );
1 }junk is just a file with random bytes. At compile-time, these get included into the binary directly before and after this meow function (there are further attempts at anti-disassembly and anti-decompilation with a the je X | jne X pattern, that always jumps to whatever X is). This one is more to confuse a human reverse engineer, which would see the strange disassembly of junk bytes during debugging.
--------------------------------------------------------------------------------------------------- threads ----
[*Thread Id:1, tid:175556] Name: "main", stopped at 0x00000040312e <NO_SYMBOL>, reason: SINGLE STEP
----------------------------------------------------------------------------------------------------- trace ----
[*#0] 0x00000040312e <NO_SYMBOL>
[ #1] 0x000000403351 <umask+0x16>
[ #2] 0x000000403b7d <NO_SYMBOL>
[ #3] 0x7ffff7dd7f77 <NO_SYMBOL>
[ #4] 0x7ffff7dd8027 <__libc_start_main+0x87>
[ #5] 0x000000402021 <NO_SYMBOL>
----------------------------------------------------------------------------------------------------------------
gef> x/20i $rip
=> 0x40312e: je 0x403133
0x403130: jne 0x403133
0x403132: call 0x34bc1336
0x403137: pop rdx
0x403138: sbb dl,BYTE PTR [rbx+rdx*2-0x59]
0x40313c: cdq
0x40313d: movabs ds:0x6529a736a4f66778,eax
0x403146: push rdi
0x403147: mov bl,0x38
0x403149: js 0x40317f
0x40314b: rex.WX adc BYTE PTR [rdi-0x540a3f2e],spl
0x403152: js 0x403127
0x403154: ins BYTE PTR es:[rdi],dx
0x403155: rex.XB (bad)
0x403157: mov bh,0x2
0x403159: mov ebp,0xd493f6a3
0x40315e: rex.X std
0x403160: sar BYTE PTR [rax],1
0x403162: in al,0xf0
0x403164: stdThis, predictably, bricks any decompiler. To be honest I really like this sort of trick, where you put in a bunch of junk bytes and watch as a disassembler really just tries its best. It is a beautiful testament to the x86 ISA that it can pass as somewhat legible assembly on first blush.
Furthermore, another anti-disassembly trick is in the .byte 0xe8 within the inline assembly in the function above. 0xe8 is the opcode for call, so a disassembler will attempt to disassemble 0xe8 + jmp rcx as a call to whatever opcode jmp rcx translates to, thereby obfuscating the indirect jmp. You can see the anti-disassembly trick in action at 0x403132. This is a known technique that I am particularly fond of. (Writing this a month or so after having set the challenge, I'm quite surprised at how many stupid anti-disassembly tricks I put in, they're all so inconsequential but I just personally find them amusing).
Fifth trick - obscure syscalls for state persistence
The main flagchecking functionality is relatively simple - some bytes of the flag are read from a buffer, then operations are performed on the read bytes, and then a comparison is made. The interesting aspect is how the comparisons are made, and how the 'success' state is accumulated over time. Simply put, there is a variable somewhere in the binary that is initially set to 0, and the difference between the computed value and the target value is bitwise or'd with that variable.
Afterwards, that variable is compared. If the computed values and the target values are all the same (i.e. the flag is correct), the variable will naturally have to be 0. If it is not, there was some difference between the two values, and the flag is wrong. The nice thing about this manner of calculation is how it cascades, and how it is difficult to manually eyeball and brute-force because of the nature of a bitwise OR. Since the check is always performed at the end, we cannot side-channel the flag-checking function through checking the number of instructions performed, and since there is no clean mapping of comparisons to bits in the final check, we cannot brute-force it byte-by-byte.
I hope the above explanation is intuitive. However, that is not the main trick being utilised. The trick is how the 'accumulator' variable is stored in the binary. It is stored as the umask of the binary, which is ordinarily used as rwx permissions. The umask is typically set through its namesake syscall, which sets the umask to the value specified in rdi, but most importantly, returns the original value of umask prior to the syscall.
So, what the binary does is call umask once to get its original value, perform the bitwise OR with the returned value, stash the umask value somewhere else, and continue onwards.
380 unsigned int umask(unsigned int mask) {
1 │ unsigned int old_mask;
2 │ __asm__(
3 │ │ "mov eax, 95\n"
4 │ │ "call sc\n"
5 │ │ :"=a" (old_mask)
6 │ │ :"D"(mask)
7 │ │ :"rcx","r11"
8 │ );
9 │
10 │ current_mask = mask;
11 │ return old_mask;The implementation above demonstrates the umask functionality. Unfortunately, reading this back, I see a pretty significant flaw, which is that current_umask is still stored in the binary. Oops! I think there is a way to not store the umask value anywhere in the binary at all. But either way, I think this is a very nice trick. It is quite easy to analyse this trick because of the existence of the strace utility, however.
Furthermore, a similar trick is performed with the comparison values loaded at each iteration of the flag-checker. Instead of the umask, we use the personality syscall. The personality syscall is a very rarely used syscall meant to dictate the behaviour of some binaries - my understanding of it is fuzzy, but I think it has something to do with ASLR. The personality syscall has an identical function to the umask for our use case, in that a call to personality sets it with our passed in argument, and returns the previous value.
So, initially, a lone personality call is performed to set the personality to the first flag-checker comparison value. Then, when the first flag-checker function fires, it calls personality to simultaneously retrieve the initial comparison value, while also setting the value for the succeeding function.
We can view this behaviour in strace, although it is not so clear.
umask(000) = 000
rt_sigreturn({mask=[]}) = 135
rt_sigreturn({mask=[]}) = 135
rt_sigreturn({mask=[]}) = 135
rt_sigreturn({mask=[]}) = 135
personality(0xbb /* PER_??? */|FDPIC_FUNCPTRS|READ_IMPLIES_EXEC|ADDR_LIMIT_32BIT|WHOLE_SECONDS|0xd0007900) = 0xf
9f8 (0xf8 /* PER_??? */|0xf900)
umask(000) = 000
rt_sigreturn({mask=[]}) = 135
rt_sigreturn({mask=[]}) = 135
rt_sigreturn({mask=[]}) = 135
rt_sigreturn({mask=[]}) = 135
personality(PER_LINUX) = 0xd2c879bb (0xbb /* PER_??? */|FDPIC_FUNCPTRS|READ_IMPLIES_EXEC|ADDR_L
IMIT_32BIT|WHOLE_SECONDS|0xd0007900)
umask(000) = 000
rt_sigreturn({mask=[]}) = 135
umask(002) = 000We can see the different calls to personality. This actually functions as a good way to isolate the personality value comparisons, although a deep understanding of the binary is required at this point to actually do anything with it.
Explanation of the core functionality
Now it is time to explain the actual way the binary functions. This took quite a lot of engineering, to be honest. A better reverse-engineer would have written an entire obfuscator and LLVM pass for this, but I am not so skilled yet and don't know how to do that. As mentioned above, essentially all of this was written in a crude cocktail of C and inline assembly. I had a lot of fun doing this.
As seen in the rt_sigreturn calls, the key is control flow obfuscation by way of calls to sigreturn. Hopefully if you are reading this you are familiar with sigreturn and I do not need to explain it, but a brief explainer is that the Linux kernel needs to know how to restore the state of a binary when it is handling a signal, so it stores all of the registers in a sigreturn frame on the stack and reads from that frame to restore registers.
We can artificially create our own sigreturn frames and call sigreturn. This is helpful as an obfuscation mechanism, as decompilers really don't know what the fuck to do with these sigreturn frames and don't know where the control flow is redirected to, and what registers are set. (Of course, how could it possibly know?)
----------------------------------------------------------------------------------------------------- stack ----
0x7ffffff32b70|+0x0000|+000: rt_sigframe.pretcode : 0x0000000000000000
0x7ffffff32b78|+0x0008|+001: rt_sigframe.uc.uc_flags : 0x0000000000000000
0x7ffffff32b80|+0x0010|+002: rt_sigframe.uc.uc_link : 0x0000000000000000
0x7ffffff32b88|+0x0018|+003: rt_sigframe.uc.uc_stack.ss_sp : 0x0000000000000000
0x7ffffff32b90|+0x0020|+004: rt_sigframe.uc.uc_stack.ss_flags|ss_size : 0x0000000000000000
0x7ffffff32b98|+0x0028|+005: rt_sigframe.uc.uc_mcontext.r8 : 0x0000000000000000
0x7ffffff32ba0|+0x0030|+006: rt_sigframe.uc.uc_mcontext.r9 : 0x0000000000000000
0x7ffffff32ba8|+0x0038|+007: rt_sigframe.uc.uc_mcontext.r10 : 0x0000000000000000
0x7ffffff32bb0|+0x0040|+008: rt_sigframe.uc.uc_mcontext.r11 : 0x0000000000000000
0x7ffffff32bb8|+0x0048|+009: rt_sigframe.uc.uc_mcontext.r12 : 0x0000000000000000
0x7ffffff32bc0|+0x0050|+010: rt_sigframe.uc.uc_mcontext.r13 : 0x0000000000000001
0x7ffffff32bc8|+0x0058|+011: rt_sigframe.uc.uc_mcontext.r14 : 0x00007ffffffeb810 -> 0x0200000fffbfbfc0 <- $r14trapframe
0x7ffffff32bd0|+0x0060|+012: rt_sigframe.uc.uc_mcontext.r15 : 0x00007ffffff32b70 -> 0x0000000000000000 <- $rsp, $r15
0x7ffffff32bd8|+0x0068|+013: rt_sigframe.uc.uc_mcontext.rdi : 0x0000000000000000
0x7ffffff32be0|+0x0070|+014: rt_sigframe.uc.uc_mcontext.rsi : 0x0000000000000000
0x7ffffff32be8|+0x0078|+015: rt_sigframe.uc.uc_mcontext.rbp : 0x00007fffffffdd60 -> 0x00007fffffffde78 -> 0x00007fffffffe1f2 -> ..
. <- $rbp
0x7ffffff32bf0|+0x0080|+016: rt_sigframe.uc.uc_mcontext.rbx : 0x0000000000000000
0x7ffffff32bf8|+0x0088|+017: rt_sigframe.uc.uc_mcontext.rdx : 0x0000000000000000
0x7ffffff32c00|+0x0090|+018: rt_sigframe.uc.uc_mcontext.rax : 0x0000000000000000
0x7ffffff32c08|+0x0098|+019: rt_sigframe.uc.uc_mcontext.rcx : 0x0000000000000000
0x7ffffff32c10|+0x00a0|+020: rt_sigframe.uc.uc_mcontext.rsp : 0x00007fffffffdb50 -> 0x0000000000403089 -> 0x20ec8348e5894855
0x7ffffff32c18|+0x00a8|+021: rt_sigframe.uc.uc_mcontext.rip : 0x0000000000404017 -> 0x57415641e5894855
0x7ffffff32c20|+0x00b0|+022: rt_sigframe.uc.uc_mcontext.rflags : 0x0000000000000202
0x7ffffff32c28|+0x00b8|+023: rt_sigframe.uc.uc_mcontext.cs|gs|fs|__pad0: 0x0000000000000032
0x7ffffff32c30|+0x00c0|+024: rt_sigframe.uc.uc_mcontext.err : 0x0000000000000000
0x7ffffff32c38|+0x00c8|+025: rt_sigframe.uc.uc_mcontext.trapno : 0x0000000000000000
0x7ffffff32c40|+0x00d0|+026: rt_sigframe.uc.uc_mcontext.oldmask : 0x0000000000000000
0x7ffffff32c48|+0x00d8|+027: rt_sigframe.uc.uc_mcontext.cr2 : 0x0000000000000000
0x7ffffff32c50|+0x00e0|+028: rt_sigframe.uc.uc_mcontext.fpstate : 0x0000000000000000
0x7ffffff32c58|+0x00e8|+029: rt_sigframe.uc.uc_mcontext.reserved[8] : 0x000000000000002b
0x7ffffff32c60|+0x00f0|+030: rt_sigframe.uc.uc_sigmask : 0x0000000000000000
0x7ffffff32c68|+0x00f8|+031: rt_sigframe.info : 0x0000000000000000Here is a helpful display of a sample sigreturn frame on the stack. We can see all the nice stuff, like stored rip values, all the other registers, so on and so forth. It's very simple to build a fake one on the stack, and I use r15 for this purpose. Initially, r15 is placed in a random area on the stack and the initial values are built. This region pointed to by r15 is the one frame consistently used throughout the entire challenge, no other frame is built elsewhere.
28 │ struct sigaction sa;
27 │ memset(&sa, 0, sizeof(sa));
26 │ sa.sa_sigaction = segfault_handler;
25 │ sigaction(SIGSEGV, &sa, ⍉);
24 │ fla();
23 │
22 │ __asm__(
21 │ │ "xor r15, r15\n"
20 │ │ "add r15, rsp\n"
19 │ │ "sub r15, 0xcafe0\n"
18 │ │ "xor r14, r14\n"
17 │ │ "add r14, rsp\n"
16 │ │ "sub r14, 0x12340\n"
15 │ │ "mov qword ptr [r15], 0\n"
14 │ │ "mov qword ptr [r15 + 0x60], r15\n"
13 │ │ "mov qword ptr [r15 + 0x58], r14\n"
12 │ │ "call set_nibble_b\n"
11 │ │ "shl rbx, 0x4\n"
10 │ │ "mov qword ptr [r15 + rbx], 0x202\n"
9 │ │ "add rbx, 0x8\n"
8 │ │ "mov qword ptr [r15 + rbx], 0x32\n"
7 │ │ "call set_nibble_e\n"
6 │ │ "shl rbx, 0x4\n"
5 │ │ "add rbx, 0x8\n"
4 │ │ "mov qword ptr [r15 + rbx], 0x2b\n"
3 │ );This shows the initialisation of the stackframe at r15. The offset at 0x50 points to the frame's corresponding R15 value, which must be persisted across each sigreturn call. 0x48 is also initialised as r14 (this will come in handy later). There are other important values required for the stackframe not to crash such as csgsfs, I will not elaborate on these.
The code snippet also shows the usage of the obfuscated register setting functions. Instead of directly loading in the frame offsets, we generate them at runtime by making calls to our set_nibble functions. I'm not sure why I put this in, to be completely honest.
r14 is stored because it is used as an array to store function pointers. Function pointers are mangled and loaded at an offset from r14, then when we need to redirect control flow to another function, we simply load the mangled function pointer from r14 and place it into the offset of r15 that corresponds to the sigreturn frame's rip field. This is the main 'dispatch' functionality. Now, a quick break to explain some further tricks:
Sixth trick - pointer mangling
21 __attribute__((always_inline))
20 inline void load_mangled(void* function_ptr, char offset) {
19 │ __asm__(
18 │ │ "movzx rax, %1\n"
17 │ │ "mov rcx, %0\n"
16 │ │ "push rcx\n"
15 │ │ "push r14\n"
14 │ │ "call set_nibble_b\n"
13 │ │ "mov rcx, rbx\n"
12 │ │ "ror qword ptr [rsp], cl\n"
11 │ │ "pop rcx\n"
10 │ │ "xor rcx, qword ptr [rsp]\n"
9 │ │ "call set_nibble_a\n"
8 │ │ "dec rbx\n"
7 │ │ "dec rbx\n"
6 │ │ "imul rax, rbx\n"
5 │ │ "mov qword ptr [r14 + rax], rcx\n"
4 │ │ "pop rax\n"
3 │ │ ::"r"(function_ptr), "r"(offset)
2 │ │ : "rax"
1 │ );
282 }I didn't want to store raw function pointers because they would be scannable in memory, so each function pointer is mangled when stored. This sort of functionality might be familiar to any experienced pwners, as it is almost lifted one-to-one from exit handler mangling. The above function takes in a function_ptr and an offset, and stores the mangled function ptr at an offset from R14.
The mangling is simple: it is a ROR of 0xb, then a XOR with the original pointer. There is some funky business going on with stack juggling in the above assembly (we push R14 onto the stack and then load from it with mov qword ptr [rsp].)
Seventh trick - swapping R14 and R15
There is another variant of the load_mangled function that operates on R15 instead of R14. This is because at different sections of the binary, the roles of R14 and R15 are swapped with a simple xchg instruction. I'm not sure why I put this in, but it's worth documenting. Here it is in action.
4 │ load_mangled(&payload_1, 0);
3 │ __asm__("xchg r14, r15");
2 │ load_mangled_r15(&win, 5);
1 │ load_mangled_r15(&lose, 10);
427 │ __asm__("xchg r14, r15");Going back to the dispatcher
There are a few more helper functions that assist with the sigreturn frame's construction. Note that we need to keep persisting the registers we want to persist across sigreturn calls, because they will get reset if we do not.
17 __attribute__((always_inline))
16 inline void load_rdi(unsigned long rdi) {
15 │ __asm__(
14 │ │ "mov qword ptr [r15 + 0x68], %q0\n"
13 │ │ "not dword ptr [r15 + 0x68]\n"
12 │ │ ::"r"(rdi)
11 │ );
10 }
9
8 __attribute__((always_inline))
7 inline void load_rsi(unsigned long rsi) {
6 │ __asm__(
5 │ │ "mov qword ptr [r15 + 0x70], %q0\n"
4 │ │ "not dword ptr [r15 + 0x70]\n"
3 │ │ ::"r"(rsi)
2 │ );
1 }These functions are inlined. An important caveat during the process of development is that you really have to fight with the compiler when you are writing inline assembly. There are a bunch of painful clobbers and inconsistencies that come hand in hand with doing bullshit like this, but that is neither here nor there.
Furthermore, there is a dedicated function set_rip() that demangles the pointer and stores it in the sigreturn frame. This is obfuscated with a bunch of pop push tricks which I honestly don't fully understand anymore - there is a lot of weird juggling for ... kind of no reason. The important thing is that this is the function that loads our next target RIP.
25 __attribute__((always_inline))
24 volatile inline void set_rip(uint64_t offset) {
23 │ __asm__(
22 │ │ "call set_nibble_a\n"
21 │ │ "shl rbx, 0x4\n"
20 │ │ "add rbx, 0x8\n"
19 │ │ "push rbx\n"
18 │ │ "mov rdx, %0\n"
17 │ │ "mov rdi, 0x8\n"
16 │ │ "imul rdx, rdi\n"
15 │ │ "mov rcx, qword ptr [r14 + rdx]\n"
14 │ │ "push r14\n"
13 │ │ "call set_nibble_b\n"
12 │ │ "push rcx\n"
11 │ │ "push rbx\n"
10 │ │ "pop rcx\n"
9 │ │ "pop rax\n"
8 │ │ "ror qword ptr [rsp], cl\n"
7 │ │ "mov rcx, rax\n"
6 │ │ "xor rcx, qword ptr [rsp]\n"
5 │ │ "pop rdx\n"
4 │ │ "pop rbx\n"
3 │ │ "mov qword ptr [r15 + rbx], rcx\n"
2 │ │ ::"r"(offset)
1 │ );
259 }The actual dispatcher function named obfuscate() that eventually calls sigreturn is defined below.
17 __attribute__((always_inline))
16 inline void obfuscate(void) {
15 │ __asm__(
14 │ │ "call set_nibble_a\n"
13 │ │ "shl rbx, 4\n"
12 │ │ "mov qword ptr [r15 + rbx], rsp\n"
11 │ │ "mov qword ptr [r15 + 0x78], rbp\n"
10 │ │ "call set_nibble_f\n"
9 │ │ "inc byte ptr [r15 + 0x50]\n"
8 │ │ "mov eax, ebx\n"
7 │ │ "mov rsp, r15\n"
6 │ │ "lea r12, [rip + 0x0e]\n"
5 │ │ "mov qword ptr [r15 - 0x2000], r12\n"
4 │ │ "jmp sigaction\n"
3 │ │ "nop\nnop\n"
2 │ );-
1 }This is another inlined function. We need to persist rsp and rbp to keep sane stack pointer + base pointer values. The inc byte ptr stuff is something I wanted to do with r13 but ended up discarding. After the sigreturn frame is prepared, we simply jmp to sigaction. The compiler treats this as a jump to the sigaction PLT stub, which we have patched earlier to point to our syscall ; ret gadget.
Note that the way we do this does not store any return addresses or anything like that. If we want to call a function and then return as per normal, instead of just chaining function calls one after the other, we need another mechanism. This is where r12 is used. Using RIP-relative addressing, we store the 'next' instruction after this inline call at a known place in the stack ([r15 - 0x2000]), so if we ever need to 'return' to the original function, we just jmp to the address stored at that position in the stack.
This is used for our arithmetic operations on the flag. There are four small stubs used, performing the operations on edi and esi.
30 __attribute__((naked))
29 void mul_byte() {
28 │ __asm__(
27 │ │ "imul edi, esi\n"
26 │ │ "jmp trampoline\n"
25 │ );
24 }
23
22 __attribute__((naked))
21 void sub_byte() {
20 │ __asm__(
19 │ │ "sub edi, esi\n"
18 │ │ "jmp trampoline\n"
17 │ );
16 }
15
14 __attribute__((naked))
13 void add_byte() {
12 │ __asm__(
11 │ │ "add edi, esi\n"
10 │ │ "jmp trampoline\n"
9 │ );
8 }
7
6 __attribute__((naked))
5 void xor_byte() {
4 │ __asm__(
3 │ │ "xor edi, esi\n"
2 │ │ "jmp trampoline\n"
1 │ );
371 }
1
2 __attribute__((naked))
3 void trampoline() {
4 │ __asm__(
5 │ │ "jmp [r15 - 0x2000]\n"
6 │ );
7 }These functions are always reached through a call to obfuscate() - i.e., indirectly, through a sigreturn call. We want to use these computations in the parent function, so we jmp back to the trampoline to handle that.
Actual flag-checking functionality
At this juncture we finally know enough about the binary to comprehend a single flag-checking function.
1 void fc_1(void) {
631 │ load_mangled(&fc_2, 31);
1 │ load_rdi(select_bytes(1));
2 │ load_rsi(select_bytes(3));
3 │ set_rip(SUB);
4 │ obfuscate();
5 │ __asm__(
6 │ │ "mov qword ptr [r15 + 0x68], rdi\n"
7 │ );
8 │ load_rsi(select_bytes(13));
9 │ set_rip(SUB);
10 │ obfuscate();
11 │
12 │ __asm__(
13 │ │ "mov dword ptr [r15 + 0x6760], edi"----
14 │ );
15 │
16 │ prepare_personality(0x351b);
17 │
18 │ __asm__(
19 │ │ "mov ebx, eax\n"
20 │ );
21 │
22 │ __asm__(
23 │ │ "not ebx\n"
24 │ │ "sub ebx, dword ptr [r15 + 0x6760]\n"
25 │ │ "or ebx, %0\n"
26 │ │ "mov edi, ebx\n"
27 │ │ "call umask\n"
28 │ │ "mov %0, eax\n"
29 │ │ ::"r"(current_mask)
30 │ );
31 │
32 │ set_rip(31);
33 │ obfuscate();
34 }Here is a single flag-checking function in its entirety. Let us dissect the steps one by one.
First, we load the next flag-checking function as a mangled pointer in our R14 table. Here, we load it at offset 31.
631 │ load_mangled(&fc_2, 31);Then, we load the flag bytes we want into the rdi and rsi fields of our sigreturn frame, and prepare the operation we want to perform on the flag bytes.
1 │ load_rdi(select_bytes(1));
633 │ load_rsi(select_bytes(3));
1 │ set_rip(SUB);Then, the obfuscate() call is triggered, firing off the sigreturn frame and performing our operation on our two selected flag byte values. Afterwards, we store the result of the arithmetic operation in the next sigreturn frame and perform another arithmetic operation.
2 │ obfuscate();
3 │ __asm__(
4 │ │ "mov qword ptr [r15 + 0x68], rdi\n"
5 │ );
6 │ load_rsi(select_bytes(13));
7 │ set_rip(SUB);
8 │ obfuscate();
9 │
10 │ __asm__(
11 │ │ "mov dword ptr [r15 + 0x6760], edi"----
12 │ );After all these calls, rdi now has our calculated value. This calculated value is stored at another position in the stack relative to r15, this time, [r15 + 0x6760].
1 │ __asm__(
644 │ │ "mov dword ptr [r15 + 0x6760], edi"----
1 │ );We now need to load our comparison value, which has been initialised in the personality value of the process. To perform this syscall to personality, we invoke this inline helper function.
13 __attribute__((always_inline))
12 inline volatile unsigned int prepare_personality(unsigned int p) {
11 │ __asm__(
10 │ │ "call set_nibble_9\n"
9 │ │ "shl rbx, 0x4\n"
8 │ │ "mov rsi, 0x87\n"
7 │ │ "mov qword ptr [r15 + rbx], rsi\n"
6 │ │ "mov qword ptr [r15 + 0x68], %q0\n"
5 │ │ ::"r"(p)
4 │ );
3 │
2 │ set_rip(3);
1 │ obfuscate();
479 }This triggers another sigreturn which redirects control flow to the 3rd function pointer stored in R14, which is another small trampoline, the call_personality function.
4 __attribute__((naked))
3 volatile void call_personality() {
2 │ __asm__("call sc\njmp [r15 - 0x2000]\n");
1 }It calls sc (a label defined within inline assembly), and then uses the jmp [r15 - 0x2000] trick to return control flow back to the caller. sc is another indirect jmp, but this is where we finally jmp to the sigaction plt handler. In summary, this is several different obfuscated jumps in and out of the binary (I, once again, am not sure why I did it like this).
- First, we invoke sigreturn to get us to call_personality.
- Second, call_personality calls the sc label.
- The sc label performs an indirect jmp via rcx to the sigaction plt entry.
- The sigaction plt entry jmps to our located syscall ; ret gadget.
- The syscall (personality) is invoked, and then the ret returns us to call_personality.
- call_personality performs an indirect jmp to the address previously stored at [r15 - 0x2000].
- Execution returns to our flagchecker function.
How fun. Continuing onwards, the output from the syscall is moved into rbx and the comparison is finally performed.
15 │ prepare_personality(0x351b);
14 │
13 │ __asm__(
12 │ │ "mov ebx, eax\n"
11 │ );
10 │
9 │ __asm__(
8 │ │ "not ebx\n"
7 │ │ "sub ebx, dword ptr [r15 + 0x6760]\n"
6 │ │ "or ebx, %0\n"
5 │ │ "mov edi, ebx\n"
4 │ │ "call umask\n"
3 │ │ "mov %0, eax\n"
2 │ │ ::"r"(current_mask)
1 │ );The result of the personality call is bitwise NOT'd, subtracted from the calculated value at [r15 + 0x6760], and bitwise OR'd with the current umask value. Then, we load another function pointer from the mangled pointer table and do it all over again.
How to solve
I won't go into depth in this section. The steps are simple(-ish):
- Isolate all of the legitimate flag-checker functions, this can be done dynamically. You can strace the binary to see which personality calls are used and then use that as a filter against all flag-checker functions that fit the function stereotype.
- Parse each valid function to see what arithmetic operations are performed on which bytes.
- Use the personality calls to gather the target values for those arithmetic operations.
- Form a list of constraints and solve with z3.
Yay!
Concluding
I don't really have a good conclusion here. I enjoyed writing the challenge and I thought it was quite hard. Thank you for reading, I hope you learned something (although I'm not sure what you would learn from this to be honest).
Reward for reaching the end of this post