Landon Fuller

On other platforms, I've found valgrind to be indispensable when it comes to discovering and fixing bugs. With Greg Parker's port of valgrind for Mac OS X now available, it's now possible to test your Mac and iPhone applications with valgrind.

Valgrind does not support ARM, but it is capable of executing i386 binaries built against the iPhone Simulator SDK. The only difficulty is that you can't run valgrind from the command line -- GUI iPhone Simulator binaries must be registered and run by Xcode, which uses the private iPhoneSimulatorRemoteClient framework to start and stop Simulator sessions.

To work around this, I wrote a small "re-exec" handler in my application's main(). It looks like this:

#define VALGRIND "/usr/local/valgrind/bin/valgrind"
 
int main(int argc, char *argv[]) {
#ifdef VALGRIND_REXEC
    /* Using the valgrind build config, rexec ourself
     * in valgrind */
    if (argc < 2 || (argc >= 2 && strcmp(argv[1], "-valgrind") != 0)) {
        execl(VALGRIND, VALGRIND, "--leak-check=full", argv[0], "-valgrind",
              NULL);
    }
#endif
  
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    int retVal = UIApplicationMain(argc, argv, nil, @"PeepsAppDelegate");
    [pool release];
    return retVal;
}

To enable this code, I added a 'valgrind' build configuration to my project which defines VALGRIND_REXEC. After switching to the valgrind configuration, my application will run in the Simulator, under valgrind:

[Session started at 2008-12-24 15:27:47 -0800.]
==38596== Memcheck, a memory error detector.
==38596== Copyright (C) 2002-2008, and GNU GPL'd, by Julian Seward et al.

Running our code under valgrind has greatly facilitated the resolution of otherwise difficult to find or entirely unknown bugs.

An update to Safari Bookbag has been posted, using the CoverFlow implementation from Peeps to replace the previous UICoverFlowLayer implementation.

Apple appears to be getting more serious about checking for private APIs -- the previous 2.2 compatibility update to Safari Bookbag was rejected by Apple due to the continued use of UICoverFlowLayer.

We're dancing over here. After quite a bumpy ride, Apple has approved Peeps for distribution in the App Store.

I even have a fancy image!

I'm already hard at work on the next version -- top requested features:

• Social network integration for fetching your friends' photographs
• Ad-hoc sharing of pictures and contact information between phones

If you've got something you'd like on the list, send us an e-mail!

Update

Peeps has been approved!

After waiting 33 days to receive word on our app, Peeps, we've got a reply:

Upon review of your application, Peeps cannot be posted to the App
Store due to the usage of a non-public API.  Usage of non-public APIs,
as outlined in the iPhone SDK Agreement section 3.3.1, is prohibited:
 
"3.3.1 Applications may only use Published APIs in the manner prescribed
by Apple and must not use or call any unpublished or private APIs. "
 
The non-public API that is included in your application comes from the
CoverFlow API set.

Let's be clear here: We did not use private API.

The last thing I would do is deliver time-bomb code to a paying customer. Private API can be broken or removed at any time by the vendor, and relying on it is unfair to your customers -- they rarely have any idea that the application they just purchased may not work next week, or next month.

So when I needed a CoverFlow-like user interface I wrote my own -- from scratch. I suppose I should be flattered that Apple mistook it for their own implementation (demo 1, demo 2).

In the mean time, I've got a support request in, and I'm waiting to hear back from the App Store. I don't fault Apple for the misunderstanding, I just wish they hadn't taken 33 days to tell me.

If you're willing to brave the App Store waters and would like to license our implementation for your own app, just say hello.

Update: You can now download a demo and purchase a copy of PLJukebox (our implementation) directly from the Plausible Labs website

Update

Peeps has been approved!

Apple's Lesson in Zen Patience

30 days ago, we were excited. Peeps 1.0 was finished, localized into a few languages, and submitted to the iPhone App Store for review. We even loved the icon — drawn by the talented Kelly J. Brownlee.

Most of my friend's apps were approved within a day, so after a week of waiting, I sent Apple an e-mail, to which they duly responded:

Your application Peeps is requiring unexpected additional time for review.
We apologize for the delay, and will update you with further status as soon as
we are able.

So we waited. An exercise in Zen — patience in the modern age. I released an update to the Plausible Database library. I finished preparing ActorKit for release. I took on some contracting work.

I sent follow-up e-mails, too. They all went unanswered. I even called Apple Developer Relations (they'll forward my query on, said the support representative. I should call for updates). So 30 days later, Peeps is still in limbo. It's not approved, nor is it rejected, it just simply is. I was fastidious in following Apple’s guidelines, used no private API, and I'm left with no idea what has triggered this state of application limbo.

What can I do? Apple doesn't answer my e-mails or phone calls, and my hard work is sitting in a queue, somewhere. I guess I write a blog post, and then try learn from this lesson in Zen.

[Addendum] - To clarify, we're not using the UICoverFlowLayer private API -- we wrote our own CoverFlow implementation (demo 1, demo 2)

The iPhone has limited memory, and even simple applications can easily trigger a low memory warning. If you've implemented caching for performance reasons, you'll often find yourself balancing memory consumption against user experience.

Measuring the current available RAM allows one to make pre-emptive decisions about memory utilization before a low memory warning is triggered, possibly avoiding overly broad cache evictions when a memory warning is triggered.

I've seen this question come up a number of times, so here's a brief code snippet demonstrating how to determine available memory from the Mach VM statistics.

#import <mach/mach.h>
#import <mach/mach_host.h>
 
static void print_free_memory () {
    mach_port_t host_port;
    mach_msg_type_number_t host_size;
    vm_size_t pagesize;
    
    host_port = mach_host_self();
    host_size = sizeof(vm_statistics_data_t) / sizeof(integer_t);
    host_page_size(host_port, &pagesize);        
 
    vm_statistics_data_t vm_stat;
              
    if (host_statistics(host_port, HOST_VM_INFO, (host_info_t)&vm_stat, &host_size) != KERN_SUCCESS)
        NSLog(@"Failed to fetch vm statistics");
 
    /* Stats in bytes */ 
    natural_t mem_used = (vm_stat.active_count +
                          vm_stat.inactive_count +
                          vm_stat.wire_count) * pagesize;
    natural_t mem_free = vm_stat.free_count * pagesize;
    natural_t mem_total = mem_used + mem_free;
    NSLog(@"used: %u free: %u total: %u", mem_used, mem_free, mem_total);
}

For more information, see the vm_statistics man page.

I occasionally use O'Reilly Publishing's Bookbag iPhone application to read my Safari Bookshelf books when I'm away from my computer (Safari Bookbag has no relation to Apple's Safari browser). Last night I was suprised to find that the Bookbag application was crashing on launch. Checking the app store application description, I found this note from O'Reilly:

This version does not work with iPhone OS v2.2. We are working on a
2.2-compatible version and will be submitting soon.

That's odd -- the most likely cause of an application breaking on a new iPhoneOS release would be if they used private API. Taking a look at the iPhone crash log, that appears to be the case:

Process:         Bookbag [2386]
[snip]
Thread 0 Crashed:
0   libSystem.B.dylib               0x31459c58 __kill + 8
1   libSystem.B.dylib               0x31459c46 kill + 4
2   libSystem.B.dylib               0x31459c3a raise + 10
3   libSystem.B.dylib               0x31474424 abort + 36
[snip]
8   libobjc.A.dylib                 0x300c1f84 objc_exception_throw + 92
9   CoreFoundation                  0x302c883e -[NSObject doesNotRecognizeSelector:] + 106
10  CoreFoundation                  0x30287222 ___forwarding___ + 490
11  CoreFoundation                  0x3026d618 _CF_forwarding_prep_0 + 40
12  Bookbag                         0x0000406c 0x1000 + 12396
13  Bookbag                         0x0000463a 0x1000 + 13882
14  Bookbag                         0x00004402 0x1000 + 13314

We can see here that Bookbag calls some method that triggers a call to -[NSObject doesNotRecognizeSelector:]. The 'doesNotRecognizeSelector:' message is sent by the Objective-C runtime when an object receives a message it can't respond to or forward. The doesNotRecognizeSelector method raises an exception, and the application terminates.

Objective-C methods don't simply disappear from an application, but they can disappear from a library. Private APIs are private because they may be modified or removed at any time -- if a method disappeared from one of the iPhoneOS libraries, chances are it was a private method.

To be sure that Bookbag really is using private API -- and to determine what API they used -- I decided to strip the DRM from the Bookbag application and disassemble their binary. Thanks to the crash log, we already know where to look -- address 0x406c, the last return address located in the Bookbag binary before doesNotRecognizeSelector: was triggered.

Apple's iPhone Application DRM utilizes Mach-O binary encryption to protect the application code. Briefly put, application decryption is handled by the kernel -- within a process space, the encrypted section of its Mach-O binary is automatically decrypted by the kernel as it is read. To strip the DRM from an iPhone app, one can simply attach a debugger to the application on a jailbroken phone, and dump the text section containing the program code. For a more comprehensive description of Mach-O binary encryption, read Amit Singh's article: Understanding Apple's Binary Protection in Mac OS X.

So after stripping the DRM (let's only use this power for good, not evil, yes?), I can disassemble the relevant Bookbag code and determine what private API is being called to trigger the crash. Here's my abridged annotated disassembly. (see unabridged)

-[CoverFlowView initWithFrame:andCount:]: 
[snip]
0x4050            1c22        mov     r2, r4          (add r2, r4, #0)
0x4052            ca0b        ldmia   r2!,{r0, r1, r3}
0x4054            c50b        stmia   r5!,{r0, r1, r3}
// argument 1 - saved pointer to @selector(initWithFrame:numberOfCovers:)
0x4056            4651        mov     r1, r10
0x4058            9b09        ldr     r3, [sp, #36]
0x405a            9a20        ldr     r2, [sp, #128]
0x405c            9311        str     r3, [sp, #68]
0x405e            9202        str     r2, [sp, #8]
0x4060            ab10        add     r3, sp, #64
0x4062            466a        mov     r2, sp
// argument 0 - UICoverFlowLayer instance
0x4064            4658        mov     r0, r11
0x4066            cb30        ldmia   r3!,{r4, r5}
0x4068            c230        stmia   r2!,{r4, r5}
// argument 2 - value of -[UIScreen frame]
0x406a            9a0e        ldr     r2, [sp, #56]
// argument 3 - number of covers
0x406c            9b0f        ldr     r3, [sp, #60]
// Call -[UICoverFlowLayer initWithFrame:numberOfCovers] (UICoverFlow is private API)
0x406e        efa8f006        blx     0xafc0  ; symbol stub for: _objc_msgSend

It's clear from the disassembly that Bookbag is using the private UICoverFlowLayer API. When Apple released iPhoneOS 2.2, the -[UICoverFlowLayer initWithFrame:numberOfCovers:] method was removed, and Safari Bookbag started crashing.

Safari Bookbag's use of UICoverFlowLayer is a discovery that has caused me some chagrin -- I held off on implementing Peeps -- which includes a fully functional CoverFlow clone -- until I saw that Apple had approved Safari Bookbag. I had assumed the authors of Bookbag implemented their own CoverFlow, too. To increase my vexation, Apple still hasn't approved Peeps nearly a month after it was submitted for review (on November 9th).

Lastly, this accutely demonstrates why one should be incredibly circumspect about using undocumented API -- it's undocumented because it can change at any time, and if it changes, your application will very likely start crashing. Even if your app slips through the cracks in Apple's review system, users will notice when your app breaks (see the Safari Bookbag reviews).

Introduction

In the process of writing Peeps, I decided to implement a library to simplify handling of background processing on the iPhone. The phone itself is a somewhat limited device, and I found that background processing was required to maintain UI interactivity while loading and rendering images for Peep's portrait and CoverFlow views.

Having spent considerable time working with actor-based concurrency systems, I decided to implement a ActorKit -- an Objective-C implementation of asynchronous inter-thread message passing. You can peruse the documentation and download the 1.0-beta1 release of ActorKit from the project page

ActorKit

ActorKit is intended to facilitate the implementation of concurrent software on both the desktop (Mac OS X) and embedded devices (iPhone OS). On the iPhone, thread-based concurrency is a critical tool in achieving high interface responsiveness while implementing long-running and potentially computationally expensive background processing. On Mac OS X, thread-based concurrency opens the door to leveraging the power of increasingly prevalent multi-core desktop computers.

To this end, ActorKit endeavours to provide easily understandable invariants for concurrent software:

• All threads are actors.
• Any actor may create additional actors.
• Any actor may asynchronously deliver a message to another actor.
• An actor may synchronously wait for message delivery from another actor.

As an actor may only synchronously receive messages, no additional concurrency primitives are required, such as mutexes or condition variables.

Building on this base concurrency model, ActorKit provides facilities for proxying Objective-C method invocations between threads, providing direct, transparent, synchronous and asynchronous execution of Objective-C methods on actor threads.

Actor Creation and Simple Message Passing

Messages

The Actor model of concurrency is fundamentally based on communication between isolated actors through asynchronous message passing. In ActorKit, any Objective-C object conforming to the NSObject protocol may be used as an inter-actor message, but message objects should be immutable to ensure thread safety. ActorKit, being written in Objective-C, can not enforce message immutablity or full isolation of Actor threads. It is entirely possible to pass mutable messages, or access mutable global variables. Like many other libraries implementing Actor message passing semantics, isolation is maintained purely through convention.

While ActorKit supports messaging with any Objective-C object, the PLActorMessage class provides generally useful facilities such as unique message transaction ids, automatically determining the message sender, and including additional message payloads.

Actor Proceses

In ActorKit, all threads are fully functioning actors -- including the "main" Cocoa thread. Each actor is represented by a PLActorProcess instance, which may be passed to any other running actors, and is used to send messages asynchronously to the given process.

ActorKit ensures that all message reception within a given actor occurs serially, and provides strict guarantees on message ordering -- messages M1 and M2 sent from actor A1 will be delivered to actor A2 in the same order. However, delivery of messages from actor A1 may be interspersed with delivery of messages sent by other actors:

In the future, ActorKit may be extended to leverage Apple's Grand Central to provide hybrid event/thread M:N scheduling of actor execution on available cores, an approach presented by Philipp Haller and Martin Odersky and implemented in Scala's Actor library (LAMP-REPORT-2007-001, EPFL, January 2007)

A Simple Echo Actor

- (void) echo {
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
    PLActorMessage  *message;
 
    // Loop forever, receiving messages
    while ((message = [PLActorKit  receive]) != nil) {
        // Echo the same message back to the sender.
        [[message sender] send: message];
 
        // Flush the autorelease pool through every loop iteration
        [pool release];
        pool = [[NSAutoreleasePool alloc] init];
    }
 
    [pool release];
}
 
- (void) run {
    // Spawn a new actor thread. This will return a process instance which may be used
    // to deliver messages the new actor.
    id proc = [PLActorKit  spawnWithTarget: self selector: @selector(echo:)];
 
    // Send a simple message to the actor.
    [proc send: [PLActorMessage  messageWithObject: @"Hello"]];
   
    // Wait for the echo
    PLActorMessage  *message = [PLActorKit  receive];
}

Sending Synchronous Messages with Actors

In an Actor system where all messages are sent asynchronously, synchronous messaging may be achieved with the following steps:

• Allocate a unique transaction id
• Send a message with that transaction id
• Wait for a reply with a matching transaction id.

Message Sequence:

ActorKit provides facilities for handling this common usage scenario. Unique transaction ids may be generated via -[PLActorKit createTransactionId] and every PLActorMessage generates and uses a new transactionId.

The PLActorRPC class utilizes the PLActorMessage's transaction id to wait for a reply on your behalf.

Send a message, and wait for the reply:

id helloActor = [PLActorKit spawnWithTarget: self selector: @selector(helloActor:)];
PLActorMessage *message = [PLActorMessage messageWithObject: @"Hello"];
PLActorMessage *reply = [PLActorRPC sendRPC: message toProcess: helloActor];

Transparently Proxying Objective-C Messages with Actors

ActorKit provides two NSProxy subclasses which provide transparent proxying of Objective-C synchronous and asynchronous method invocations via actor messaging. PLActorRPCProxy spawns a new actor to execute Objective-C methods for a given object instance, while PLRunloopRPCProxy executes Objective-C methods on a provided NSRunLoop.

In combination, these classes allow for safely and transparenty executing methods on Objective-C instances from any thread:

NSString *actorString = [PLActorRPCProxy proxyWithTarget: @"Hello"];
NSString *runloopString = [PLRunloopRPCProxy proxyWithTarget: @"Hello" runLoop: [NSRunLoop mainRunLoop]];
 
// Executes synchronously, via a newly created actor thread.
[actorString description];
 
// Executes synchronously, on the main runloop.
[runloopString description];

By default, PLActorRPCProxy and PLRunloopRPCProxy will execute methods synchronously, waiting for completion prior to returning. In order to execute a method asynchronously -- allowing a long running method to execute without waiting for completion -- it is necessary to mark methods for asynchronous execution.

The Objective-C runtime provides a number of type qualifiers that were intended for use in implementing a Distributed Object system. Of particular note is the 'oneway' qualifier, which allows us to specify that a method should be invoked asynchronously.

When a method is declared with a return value of 'oneway void', the proxy classes will introspect this return value, and execute the method asynchronously, without waiting for a reply:

- (oneway void) asyncMethod {
    // Execute, asynchronously
}

- (NSString *) synchronousMethod {
    // Execute, synchronously
    return @"Hello";
}

A Simple Echo Actor with PLActorRPCProxy

The following actor returns a proxy from its init method, ensuring that all methods called on the object instance will occur via the actor thread.

// An actor that responds to Objective-C messages either synchronously or asynchronously.
@implementation EchoActor
 
- (id) init {
    if ((self = [super init]) == nil)
        return nil;
  
    // Launch our actor
    id proxy = [[PLActorRPCProxy  alloc] initWithTarget: self];
  
    // Release ourself, as the proxy has retained our object,
    // and return our proxy to the caller
    [self release];
    return proxy;
}
  
// Method is called asynchronously
- (oneway void) asynchronousEcho: (NSString *) text listener: (EchoListener *) echoListener {
    [echoListener receiveEcho: text];
}
  
// Method is called synchronously
- (NSString *) synchronousEcho: (NSString *) text {
    return text;
}
  
@end

Integration & Development Services

ActorKit is provided free of charge under the MIT license, and may be freely integrated with any application. We can provide assistance with integrating our code in your own iPhone or Mac application, as well as development of additional features under a license of your choosing. We're also available for standard iPhone and Mac OS X application development.

Contact me for more information, or visit the Plausible Labs website.

References

Actors that Unify Threads and Events, Philipp Haller and Martin Odersky, LAMP-REPORT-2007-001, EPFL, January 2007. Available from http://lamp.epfl.ch/~phaller/actors.html.

Introduction

Due to Apple's restrictions, third party apps for the iPhone may not include embedded dynamic frameworks or libraries, necessitating the use of static libraries for implementing shared code.

To make things more difficult, iPhone projects are actually targeted at two distinct platforms: iPhoneOS (the phone) and iPhoneSimulator (the simulator). The two platforms are very different, and one can't build a universal binary for both the simulator and the phone by specifying both i386 and armv6 architectures, as you would if building a universal binary for Mac OS X.

These issues complicate shipping re-usable libraries for the iPhone, and I've struggled to find a reasonable method for releasing Plausible Database, as well as the number of other open source libraries I plan on releasing with our (yet to be approved) iPhone application, Peeps.

Now that the NDA is no longer in effect (yay), this article is intended to be the first in a (likely sporadic) series on iPhone development and to-be-released open source libraries.

Frameworks are Good

A significant advantage of frameworks over simple static libraries is the bundling of headers, the library, and any resources into an easy to install bundle -- to use a framework, simply copy it into your project and add it to your target. The include and linker paths will be automatically configured, and resource references will even be resolved to the correct file path.

One doesn't have to give up all of functionality of frameworks when developing for the iPhone -- one undocumented (and, to quote, "semi-supported") feature of framework linking is "static frameworks". Generally a framework includes a dylib -- "MyCode.framework/Versions/Current/MyCode". By replacing this dylib with a static library, the framework can be easily imported and used within an iPhone project.

Static frameworks are not a full replacement for standard frameworks -- standard methods for accessing the framework resources, such as +[NSBundle bundleForClass:], will not return the expected bundle. The bundle path is determined by querying dyld for the library path corresponding to the symbol address -- In the case of static linking, this will return the path to the binary into which the static library was linked. (see the CFBundle implementation (APSL, requires free ADC account)). Additionally, NSBundle/CFBundle will examine the previous stack from for the return address, and use that return address to determine the calling code's dyld binary path. Given these restrictions, some work-arounds are possible, such as using -[NSBundle privateFrameworkPath] to locate the path to the embedded framework.

Additionally, the iPhoneOS and iPhoneSimulator SDK specifications do not support the 'Framework' product type, meaning that the framework directory structure must be built by hand. Fortunately, frameworks are fairly simple directory structure. For Plausible Database, I simply the built Mac OS X framework, and then replacing the enclosed dynamic library with the iPhone static library. (To see where supported build types are defined, take a look at /Developer/Platforms/iPhoneOS.platform/Developer/Library/Xcode/Specifications/iPhoneOSProductTypes.xcspec on your development system.)

While static frameworks are fairly limited (by a significant margin), I've found them to be a convenient, if small, improvement on shipping standalone static libraries. Your mileage may vary.

Universal Binaries: iPhoneOS vs. iPhoneSimulator

While Mac OS X bundles (applications, etc) include support for multiple platforms:

Application.app/Contents/MacOS
Application.app/Contents/Windows

frameworks do not:

MyFramework.framework/Versions/A/MyFramework

As noted earlier, iPhone projects actually target two distinct platforms -- the simulator, and the phone.

It is possible, if incorrect, to lipo together a single static library that may be used for both platforms. With such a binary, the i386 binary will be used when linking against the simulator SDK, and the armv6 binary will be used when linking against the iphoneos SDK. While this will work without error, it may break in the future. For example, if Apple ever releases an i386 based iPhone (using, for instance, the Intel Atom, then binaries intended to be used with the iPhoneOS SDK must be built armv6/x86 universal, and the result will not work when used with the simulator.

As an alternative, one can use the PLATFORM_NAME or EFFECTIVE_PLATFORM_NAME to link against a platform-specific static library; simply add -lSomeLibrary${EFFECTIVE_PLATFORM_NAME} to your target's linker options.

Conclusion

As it stands, shipping librares for the iPhone is fairly messy. Until a more approachable mechanism is provided for shipping iPhone libraries, I plan on building universal i386-simulator/armv6-iphoneos static frameworks for my open source projects and internal libraries. I might change my mind later.

In the meantime, I've filed rdar://6413456 -- RFE: Support for dylibs (and embedded frameworks) in 3rd-party iPhone apps. I'm sure it's a duplicate. =)