PT_DENY_ATTACH is a non-standard ptrace() request type that prevents a debugger from attaching to the calling process. Adam Leventhal recently discovered that Leopard extends PT_DENY_ATTACH to prevent introspection into processes using dtrace. I hope Adam will forgive me for quoting him here, but he put it best:
This is antithetical to the notion of systemic tracing, antithetical to the goals of DTrace, and antithetical to the spirit of open source. I'm sure this was inserted under pressure from ISVs, but that makes the pill no easier to swallow.
This article will cover disabling PT_DENY_ATTACH for all processes on Mac OS X 10.5. Over the previous few years, I've provided similar hacks for both Mac OS X 10.4, and 10.3.
To be clear: this work-around is a hack, and I hold that the correct fix is the removal of PT_DENY_ATTACH from Mac OS X.
In xnu the sysent array includes function pointers to all system calls. By saving the old function pointer and inserting my own, it's relatively straight-forward to insert code in the ptrace(2) path.
However, with Mac OS X 10.4, Apple introduced official KEXT Programming Interfaces, with the intention of providing kernel binary compatibility between major operating system releases. As a part of this effort, the sysent array's symbol can not be directly resolved from a kernel extension, thus removing the ability to easily override system call. In 10.4, I was able to work-around this with the amusing temp_patch_ptrace() API. This API has disappeared in 10.5.
For Leopard, I decided to find a public symbol that is placed in the data segment, nearby the sysent array. In the kernel's data segment, nsysent is placed (almost) directly before the sysent array. By examining mach_kernel I can determine the offset to the actual sysent array, and then use this in my kext to patch the actual function. To keep things safe, I added sanity checks to verify that I'd found the real sysent array.
Each sysent structure has the following fields:
struct sysent {
int16_t sy_narg; /* number of arguments */
int8_t reserved; /* unused value */
int8_t sy_flags; /* call flags */
sy_call_t *sy_call; /* implementing function */
sy_munge_t *sy_arg_munge32; /* munge system call arguments for 32-bit processes */
sy_munge_t *sy_arg_munge64; /* munge system call arguments for 64-bit processes */
int32_t sy_return_type; /* return type */
uint16_t sy_arg_bytes; /* The size of all arguments for 32-bit system calls, in bytes */
};
The "sy_call" field contains a function pointer to the actual implementing function for a given syscall. If we look at the actual sysent table, we'll see that the first entry is "SYS_nosys":
__private_extern__ struct sysent sysent[] = {
{0, 0, 0, (sy_call_t *)nosys, NULL, NULL, _SYSCALL_RET_INT_T, 0},
To narrow down the haystack, we'll find the address of the nsysent variable, and then search for the nosys function pointer -- as shown above, nosys should be the first entry in the sysent array.
nm /mach_kernel| grep _nsysent 00502780 D _nsysent
nm /mach_kernel| grep T\ _nosys 00388604 T _nosys
Here is a dump of the mach_kernel, starting at 0x502780. You can see the value is 0x01AB, or 427 -- by looking at the kernel headers, we can determine that this is the correct number of syscall entries. 33 bytes after nsysent, we see 0x388604 (in little-endian byte order) -- this is our nosys function pointer. After counting the size of the sysent structure fields, we can determine that the the sysent array is located 32 bytes after the nsysent variable address. (On PPC, it's directly after).
otool -d /mach_kernel 00502780 ab 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00502790 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 005027a0 00 00 00 00 04 86 38 00 00 00 00 00 00 00 00 00
Once we have the address of the array, we can find the SYS_ptrace entry and substitute our own ptrace wrapper:
static int our_ptrace (struct proc *p, struct ptrace_args *uap, int *retval)
{
if (uap->req == PT_DENY_ATTACH) {
printf("[ptrace] Blocking PT_DENY_ATTACH for pid %d.\n", uap->pid);
return (0);
} else {
return real_ptrace(p, uap, retval);
}
}
kern_return_t pt_deny_attach_start (kmod_info_t *ki, void *d) {
...
real_ptrace = (ptrace_func_t *) _sysent[SYS_ptrace].sy_call;
_sysent[SYS_ptrace].sy_call = (sy_call_t *) our_ptrace;
...
}
You can download the kext source here (sig).
Buyer beware: This code has only seen limited testing, and your mileage may vary. If something goes wrong, sanity checks should prevent a panic, and the module will fail to load.
If the module loads correctly, you should see the following in your dmesg output:
[ptrace] Found nsysent at 0x502780 (count 427), calculated sysent location 0x5027a0. [ptrace] Sanity check 0 1 0 3 4 4: sysent sanity check succeeded. [ptrace] Patching ptrace(PT_DENY_ATTACH, ...). [ptrace] Blocking PT_DENY_ATTACH for pid 82248.
Note: To access the nsysent symbol, the kext is required to declare a dependency on a specific version of Mac OS X. When updating to a new minor release, it should be sufficient to change the 'com.apple.kernel' version in the kext's Info.plist. I've uploaded a new version of the kext with this change, but I won't provide future updates unless a code change is required.
<key>OSBundleLibraries</key>
<dict>
<key>com.apple.kernel</key>
<string>9.2.0</string>
</dict>
Much thanks to Ryan Chapman for noting this issue, and testing the kext with 10.5.2.