Landon Fuller

On Mac OS X, application signing entitlements enforce client-side constraints on non-AppStore (or non-Apple-distributed) applications. This is used, for instance, to prevent a non-AppStore application from using the MapKit APIs.

Recently, I wanted to backport Xcode 6.3 from Yosemite, getting it running on Mavericks. Unfortunately, simply stripping signatures wasn't an option -- Xcode itself transitively depends on code signing via its use of XPC.

It's also not possible to simply resign a modified Xcode binary with a local adhoc certificate (or a standard paid Mac Developer certificate); Xcode relies on Apple-privileged entitlements -- including the MapKit entitlement -- that aren't available without a trusted entitlement-granting provisioning profile.

These AppStore-only functionality constraints are enforced by the /usr/libexec/taskgated daemon; to work around it, I implemented task-unchain -- a small binary patch that may be applied to taskgated, disabling all checks for restricted entitlements.

Running 10.4.11 under VMWare Fusion 6

Introduction

I recently agreed to port a small piece of code to Tiger (10.4), assuming that it would be relatively easy to bring up Tiger up in VMWare Fusion on my Mavericks desktop; I already keep VMs for all of the other Mac OS X releases starting from 10.6.

As it turns out, it wasn't so easy; after downloading a copy of 10.4 Server from the Apple Developer site, I found that the installer DVD would panic immediately on boot. Apparently Tiger uses uses the older CPUID[2] mechanism for fetching processor cache and TLB information (see xnu's set_intel_cache_info() and page 3-171 of the Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 2).

Virtualized systems query the hosts' real CPUID data to acquire basic information on supported CPU futures; newer processors -- mine included -- do not include cache descriptor information in CPUID[2]. Unable to read CPUID[2], the Tiger kernel left the cache line size initialized to 0 in set_intel_cache_info(), and then panicked immediately thereafter when initializing the commpage and checking for (and not finding) a required 64 byte cache line size.

In theory, VMware Fusion has support for masking CPUID values for compatibility with the guest OS by editing the .vmx configuration by hand, but I wasn't able to get any cpuid mask values to actually apply, so I eventually settled for binary patching the Tiger kernel to force proper initialization of the cache_linesize value.

If you find yourself needing to run Tiger for whatever reason, I've packaged up my patch as a small command line tool. It successfully applies against the 10.4.7 and 10.4.11 kernels that I tried. While you should run it against /mach_kernel after mounting the Tiger file system under your host OS, I found that tool worked just fine just fine when run directly against my raw VMware disk image file, as well as against the Mac OS X Server 10.4.7 installation DMG.

Using the tool, I was able to install 10.4.7, upgrade to 10.4.11, and then patch the 10.4.11 VMDK to allow booting into the upgraded system.

Source Code

You can download the source here. Obviously, this is totally unsupported and could destroy all life as we know it, or just your Tiger VM; use at your own risk.

It took me almost 13 years, but I finally sat down and solved a problem that has annoyed me since Mac OS X 10 Public Beta: synchronizing OpenSSL's trusted certificate authorities with Mac OS X's trusted certificate authorities.

The issue is simple enough; certificate authorities are used to verify the signatures on x509 certificates presented by servers when you connect over SSL/TLS. Mac OS X's SSL APIs have one set of certificates, OpenSSL has another set, and OpenSSL doesn't ship any by default.

The result of this certificate bifurcation:

• Any app using SSL (curl, svn, git) will fail to validate remote certificates unless you manually add all of the CA certificates to OpenSSL's certificate store.
• If you use a custom CA (eg, to sign certificates for your internal services), you'll need to add the CA certificate to OpenSSL's certificate store.

Given that computers can automate the mundane, I finally put aside some time to write certsync (permanent viewvc).

It's a simple, single-source-file utility that reads all trusted CAs via Mac OS X's keychain API, and then spits them out as an OpenSSL PEM file. If you pass it the -s flag, it will continue to run after the first export, automatically regenerating the output file when the system trust settings or certificate authorities are modified.

I wrote the utility for use in MacPorts; it's currently available as an experimental port. If you'd like to test it out, it can be installed via:

port -f deactivate curl-ca-bundle
port install certsync
port load certsync # If you want the CA certificates to be updated automatically

Note that forcibly deactivating ports is not generally a good idea, but you can easily restore your installation back to the expected configuration via:

port uninstall certsync
port activate curl-ca-bundle

Or, you can build and run it manually:

clang -mmacosx-version-min=10.6 certsync.m -o certsync -framework Foundation -framework Security -framework CoreServices
./certsync -s -o /System/Library/OpenSSL/cert.pem

I also plan to investigate support for Java JKS truststore export (preferably before 2026).

Introduction

Some months ago, I picked up a an Aeon Labs Z-Stick Series 2. The Z-Stick is USB-equipped Z-Wave controller, using CP210x chipset to provide access to the Z-Wave module's UART over USB. The official driver from Silicon Labs simply didn't work at all, timing out on any read or write to the device. So I decided to take a crack at writing my own driver.

The driver works great for my purposes -- I can turn my lights on and off without ever leaving my desk. Awesome, right?

The bad news? That's where I'm going to stop, at least for now. Between poor documentation, ambiguously defined behavior, and more layers of indirection than I can reasonably decipher in a spare afternoon, I've had my fill of IOKit for a while. Of the 1.2k lines of code in the driver so far, maybe 40 LoC actually has anything to do with talking to the CP210x; the rest is all IOKit-required boilerplate.

Adventures in IOKit

In my experience, third-party USB serial drivers for Mac OS X tend to be extremely buggy. Most of the drivers I've tested panic during use, don't work at all, or fail under relatively standard usage such as unplugging the device while running. I previously assumed that the entire blame for this rested on the driver's authors. Then I tried to write a USB serial driver for Mac OS X, and gained a whole new appreciation for anyone embarking on this journey.

If I were to revisit this in the future, I'd like to spend some time abstracting out the driver into a generic re-usable USB serial layer that I could use to implement additional drivers. Most USB serial devices have nearly identical interfaces -- a bulk input pipe, a bulk output pipe, and actual control requests that vary between devices, but tend to provide similar functionality -- a good fit for centralizing all the complex IOKit state management and boilerplate in one place. This is what Haiku does, to pretty good effect: compare the roughly 138 lines of Haiku's implementation with my IOKit monster

Getting the Source

If you'd like to use the driver -- or help with the remaining issues -- you can find the source available at the mac-cp210x project page. The additional work will be solving lifecycle issues around asynchronous device control messages and unplug/driver unload behavior, enabling the existing code to allow manual control over RTS/DTR lines, and implementing state updates on changes to the incoming control lines.

Otherwise, I'll probably revisit the code whenever spending a few weekends with IOKit sounds like fun again.

Introduction

Five months ago, CVE-2008-5353 and other vulnerabilities were publicly disclosed, and fixed by Sun.

CVE-2008-5353 allows malicious code to escape the Java sandbox and run arbitrary commands with the permissions of the executing user. This may result in untrusted Java applets executing arbitrary code merely by visiting a web page hosting the applet. The issue is trivially exploitable.

Unfortunately, these vulnerabilities remain in Apple's shipping JVMs, as well as Soylatte 1.0.3. As Soylatte does not provide browser plugins, the impact of the vulnerability is reduced. The recent release of OpenJDK6/Mac OS X is not affected by CVE-2008-5353.

Work-Arounds

• Mac OS X users should disable Java applets in their browsers and disable 'Open "safe" files after downloading' in Safari.
• Soylatte users running untrusted code should upgrade to an OpenJDK6-based release, where possible. No future releases of the JRL-based Soylatte branch are planned at this time. If this is an issue for you, please feel free to contact me.

Patch

Update 06-15-2009: Apple has released Java for Mac OS X 10.5 Update 4, which contains a fix for this issue.

To update your system, run "Software Update" from the Apple menu.

Note: Safari users should leave 'Open "safe" files after download' permanently disabled. Similarly critical vulnerabilities unrelated to Java remain in Safari's handling of "Safe" files.

Proof of Concept

Unfortunately, it seems that many Mac OS X security issues are ignored if the severity of the issue is not adequately demonstrated. Due to the fact that an exploit for this issue is available in the wild, and the vulnerability has been public knowledge for six months, I have decided to release a my own proof of concept to demonstrate the issue.

If you visit the following page, "/usr/bin/say" will be executed on your system by a Java applet, with your current user permissions. This link will execute code on your system with your current user permissions. The proof of concept runs on fully-patched PowerPC and Intel Mac OS X systems.

Credit

Thanks to Jeffrey Czerniak for bringing this issue to my attention.

Update: Julien Tinnes e-mailed with a link to his in-depth discussion regarding the vulnerability available here.

Introduction

PT_DENY_ATTACH is a non-standard ptrace() request type that prevents a debugger from attaching to the calling process. Adam Leventhal recently discovered that Leopard extends PT_DENY_ATTACH to prevent introspection into processes using dtrace. I hope Adam will forgive me for quoting him here, but he put it best:

This is antithetical to the notion of systemic tracing, antithetical to the goals of DTrace, and antithetical to the spirit of open source. I'm sure this was inserted under pressure from ISVs, but that makes the pill no easier to swallow.

This article will cover disabling PT_DENY_ATTACH for all processes on Mac OS X 10.5. Over the previous few years, I've provided similar hacks for both Mac OS X 10.4, and 10.3.

To be clear: this work-around is a hack, and I hold that the correct fix is the removal of PT_DENY_ATTACH from Mac OS X.

How it Works

In xnu the sysent array includes function pointers to all system calls. By saving the old function pointer and inserting my own, it's relatively straight-forward to insert code in the ptrace(2) path.

However, with Mac OS X 10.4, Apple introduced official KEXT Programming Interfaces, with the intention of providing kernel binary compatibility between major operating system releases. As a part of this effort, the sysent array's symbol can not be directly resolved from a kernel extension, thus removing the ability to easily override system call. In 10.4, I was able to work-around this with the amusing temp_patch_ptrace() API. This API has disappeared in 10.5.

For Leopard, I decided to find a public symbol that is placed in the data segment, nearby the sysent array. In the kernel's data segment, nsysent is placed (almost) directly before the sysent array. By examining mach_kernel I can determine the offset to the actual sysent array, and then use this in my kext to patch the actual function. To keep things safe, I added sanity checks to verify that I'd found the real sysent array.

Each sysent structure has the following fields:

struct sysent {
	int16_t		sy_narg;		/* number of arguments */
	int8_t		reserved;		/* unused value */
	int8_t		sy_flags;		/* call flags */
	sy_call_t	*sy_call;		/* implementing function */
	sy_munge_t	*sy_arg_munge32;	/* munge system call arguments for 32-bit processes */
	sy_munge_t	*sy_arg_munge64;	/* munge system call arguments for 64-bit processes */
	int32_t		sy_return_type; /* return type */
	uint16_t	sy_arg_bytes;	/* The size of all arguments for 32-bit system calls, in bytes */
};

The "sy_call" field contains a function pointer to the actual implementing function for a given syscall. If we look at the actual sysent table, we'll see that the first entry is "SYS_nosys":

__private_extern__ struct sysent sysent[] = {
    {0, 0, 0, (sy_call_t *)nosys, NULL, NULL, _SYSCALL_RET_INT_T, 0},

To narrow down the haystack, we'll find the address of the nsysent variable, and then search for the nosys function pointer -- as shown above, nosys should be the first entry in the sysent array.

nm /mach_kernel| grep _nsysent
00502780 D _nsysent

nm /mach_kernel| grep T\ _nosys
00388604 T _nosys

Here is a dump of the mach_kernel, starting at 0x502780. You can see the value is 0x01AB, or 427 -- by looking at the kernel headers, we can determine that this is the correct number of syscall entries. 33 bytes after nsysent, we see 0x388604 (in little-endian byte order) -- this is our nosys function pointer. After counting the size of the sysent structure fields, we can determine that the the sysent array is located 32 bytes after the nsysent variable address. (On PPC, it's directly after).

otool -d /mach_kernel
00502780        ab 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00502790        00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
005027a0        00 00 00 00 04 86 38 00 00 00 00 00 00 00 00 00

Once we have the address of the array, we can find the SYS_ptrace entry and substitute our own ptrace wrapper:

static int our_ptrace (struct proc *p, struct ptrace_args *uap, int *retval)
{
	if (uap->req == PT_DENY_ATTACH) {
		printf("[ptrace] Blocking PT_DENY_ATTACH for pid %d.\n", uap->pid);
		return (0);
	} else {
		return real_ptrace(p, uap, retval);
	}
}

kern_return_t pt_deny_attach_start (kmod_info_t *ki, void *d) {
	...
	real_ptrace = (ptrace_func_t *) _sysent[SYS_ptrace].sy_call;
	_sysent[SYS_ptrace].sy_call = (sy_call_t *) our_ptrace;
	...
}

Download

You can download the kext source here (sig).

Buyer beware: This code has only seen limited testing, and your mileage may vary. If something goes wrong, sanity checks should prevent a panic, and the module will fail to load.

If the module loads correctly, you should see the following in your dmesg output:

[ptrace] Found nsysent at 0x502780 (count 427), calculated sysent location 0x5027a0.
[ptrace] Sanity check 0 1 0 3 4 4: sysent sanity check succeeded.
[ptrace] Patching ptrace(PT_DENY_ATTACH, ...).
[ptrace] Blocking PT_DENY_ATTACH for pid 82248.

Note: To access the nsysent symbol, the kext is required to declare a dependency on a specific version of Mac OS X. When updating to a new minor release, it should be sufficient to change the 'com.apple.kernel' version in the kext's Info.plist. I've uploaded a new version of the kext with this change, but I won't provide future updates unless a code change is required.

<key>OSBundleLibraries</key>
<dict>
    <key>com.apple.kernel</key>
    <string>9.2.0</string>
</dict>

Much thanks to Ryan Chapman for noting this issue, and testing the kext with 10.5.2.

Minor Bugfix

This release fixes a name resolution bug reported by Leif Nelson of LLNL.

• If java.net.preferIPv4Stack is set to true, getaddrinfo() makes (time consuming) MDNS SRV requests

I tracked this down to this copy/paste bug in resolver code:

error = getaddrinfo(hostname, "domain", &hints, &res);

The service argument should have been NULL.

Download

Binaries, source, build, and contribution instructions are all available from SoyLatte Project Page

This was originally sent to the OpenJDK Porters Development List, and reproduced here so I can shamelessly call for volunteers. If you've a desire to work on Mac OS X Java Integration, your help is always appreciated. Send me an e-mail, or join the OpenJDK porters group!

AWT Native Toolkit

I've spent a bit of time working on a native AWT toolkit. My initial tasks have been:

• Running the Cocoa event loop on the 'main' (first) thread
• Getting a window on the screen
• Drawing to the window

The first two items are done -- It's possible to display a basic AWT window (CPanelPeer, CWindowPeer), but not draw to it.

In pursuit of drawing, I've begun work on implementing a MacGraphicsDevice and a MacGraphicsEnvironment -- my current plan is to leverage the existing java2d OpenGL pipeline, if possible. I'm still getting my head around the best approach to this, especially in relation to resolution independence on Leopard.

If you'd like to check out the work, it's available via the development mercurial repository:

http://hg.bikemonkey.org/javasrc_1_6_jrl_darwin/

The code is currently located in:

j2se/src/solaris/classes/sun/awt/mac/
j2se/src/solaris/native/sun/awt/mac/

I will probably need to move the Mac-specific code out of the j2se/src/solaris directory. One of my goals is to retain support for both the X11 and the CG/Cocoa AWT toolkits.

The build only AWT, you can cd to j2se/make/sun/awt and run:

make ALT_BOOTDIR=/System/Library/Frameworks/JavaVM.framework/Home \
ALT_MOTIF_DIR=/opt/local SYS_CFLAGS="" LANG="C" JAVA_HOME="" CLASSPATH="" \
LD_LIBRARY_PATH="" MAKEFLAGS="" SKIP_COMPARE_IMAGES="YES" \
BUILD_DEPLOY="false" ALT_DEVTOOLS_PATH=/usr ALT_CUPS_HEADERS_PATH=/usr/include \
HOTSPOT_BUILD_JOBS=5 PARALLEL_BUILD_JOBS=5 \
ALT_HOTSPOT_CLIENT_PATH=/usr/local/soylatte16-i386-1.0/jre/lib/i386/client/ \
ALT_HOTSPOT_SERVER_PATH=/usr/local/soylatte16-i386-1.0/jre/lib/i386/server/

This will output a working JVM in j2se/build/bsd-i586. For the initial build, you make need to run make from top-level j2se/make directory.

Merge to BSD Repository

I've completed the merge of the Mac OS X port to the BSD Java project. With this work done, I've split Mac OS X development into two branches:

• Development - Integration with Mac OS X-specific technology (native AWT, etc.)
• Stable - A baseline Mac OS X port without additional native integration. Synchronized with the BSD repository.

If you'd like information on accessing the Mercurial repositories, check out the SoyLatte Project Page

General Release

After several weeks of using SoyLatte for my own development, and the testing of an exceptional group of early adopters, it's time to issue a 1.0 release. This release provides:

• A stable JVM, usable for real-world development
• The benefits of Java 6 performance enhancements
• X11 Swing (Nothing is perfect!)

Download Information

To download SoyLatte 1.0, please see the SoyLatte Project Page. If you are interested in contributing, please consider joining the OpenJDK Porter's Mailing List to say hello!

Changes since DP3

The only change in the 1.0 release is the acceptance of the -XstartOnFirstThread option. Nearly all outstanding changes have been merged upstream into the BSD Java project, and I have split development into two branches -- a stable branch, and an unstable branch. It's my intention (and I have begun work on) integrating Mac OS X-specific functionality within the unstable development branch.

Java Posse

The exceptionally nice guys from the Java Posse podcast called me up for an interview about Java 6 on Mac OS X. You can hear me sounding like a goof here.

I've also been awarded the first-ever Java Posse award. Free beer! Cheers to that!

The BSD Java Project deserves considerable credit as well. The Mac OS X Java port is dependent on the work of amazing people like Greg Lewis (FreeBSD) and Kurt Miller (OpenBSD), and I'll happily buy *them* a beer.

As the podcast states, if you're interested in working on native graphics for Mac OS X, sound, or anything else you think needs doing, check out the project page and join in!

OpenJDK Porters Group

I'm very happy to report that the OpenJDK Project's Porters Group has been established!

While the OpenJDK code base covers some mainstream CPU architectures and operating systems, there are many more platforms that are attractive targets for porting efforts for Java. This group exists to bundle and aid such efforts under the OpenJDK umbrella, and integrate them in the OpenJDK community through porting projects.

The BSD porting projects, led by Kurt Miller and Greg Lewis, have expressed interest in joining OpenJDK, as well as the Mac OS X porting project led by Landon Fuller.

This is fantastic -- many thanks to the OpenJDK developers for putting this together. I hope to move development work over to OpenJDK in the near(ish?) future.