Interview with The Java Posse, OpenJDK Porters Group

29 Nov 2007, 20:10 PST

Java Posse

The exceptionally nice guys from the Java Posse podcast called me up for an interview about Java 6 on Mac OS X. You can hear me sounding like a goof here.

I've also been awarded the first-ever Java Posse award. Free beer! Cheers to that!

The BSD Java Project deserves considerable credit as well. The Mac OS X Java port is dependent on the work of amazing people like Greg Lewis (FreeBSD) and Kurt Miller (OpenBSD), and I'll happily buy *them* a beer.

As the podcast states, if you're interested in working on native graphics for Mac OS X, sound, or anything else you think needs doing, check out the project page and join in!

OpenJDK Porters Group

I'm very happy to report that the OpenJDK Project's Porters Group has been established!

While the OpenJDK code base covers some mainstream CPU architectures and operating systems, there are many more platforms that are attractive targets for porting efforts for Java. This group exists to bundle and aid such efforts under the OpenJDK umbrella, and integrate them in the OpenJDK community through porting projects.

The BSD porting projects, led by Kurt Miller and Greg Lewis, have expressed interest in joining OpenJDK, as well as the Mac OS X porting project led by Landon Fuller.

This is fantastic -- many thanks to the OpenJDK developers for putting this together. I hope to move development work over to OpenJDK in the near(ish?) future.

[/code/macosx] — permanent link

SoyLatte: Developer Preview Release 3

28 Nov 2007, 03:22 PST

Now with more Soy

Thanks to all for testing Release 2! I'm pleased to announce Developer Preview Release 3, which includes compatibility and feature improvements based on the latest round of community suggestions. I hope you don't mind the rapid release cycle -- I figured that it's important to iterate quickly with the initial testing releases.

New Features & Compatibility Fixes

Binaries, source, build, and contribution instructions are all available from the new SoyLatte Project Page -- my blog posts were starting to get a little long!

[/code/macosx] — permanent link

Java 6 Port: Developer Preview Release 2 for Leopard and Tiger

22 Nov 2007, 16:50 PST

Community Process

I'd like to thank everyone who has downloaded, tested, and reported bugs. The result is "Developer Preview Release 2" -- Thanks!

If you haven't already, consider downloading a copy and running your software. Let me know how it works out for you!

Changes since Release 1

Features

Bug fixes

Fetching and Building from Source

Building

The build has been simplified. To build the 32-bit JVM:

cd control/make
make ALT_BOOTDIR=/System/Library/Frameworks/JavaVM.framework/Home \
ALT_MOTIF_DIR=/opt/local SYS_CFLAGS="" LANG="C" JAVA_HOME="" CLASSPATH="" \
LD_LIBRARY_PATH="" MAKEFLAGS="" SKIP_COMPARE_IMAGES="YES" \
BUILD_DEPLOY="false" ALT_DEVTOOLS_PATH=/usr ALT_CUPS_HEADERS_PATH=/usr/include \
HOTSPOT_BUILD_JOBS=1 PARALLEL_BUILD_JOBS=1

The following additional make options are available:

To build with extra debugging code and symbols, use the "debug_build" target.

To target Tiger, set the MACOSX_DEPLOYMENT_TARGET environmental variable to "10.4", and pass DARWIN_SDK=/Developer/SDKs/MacOSX10.4u.sdk as a make flag. The Tiger SDK currently must be build on Leopard, due to bugs in the 10.4 compiler's -mstackrealign code generation. This is on the long-term to-be-fixed list.

Fetching the source

Sources are available as a downloadable archive, or from a mercurial repository. To download the source code, you must be a licensee in good standing under the Java Research License. To ensure compliance, downloading the source requires authentication:

Username: 'jrl' and Password: 'I am a Licensee in good standing'

Download: jdk6_devpreview_r2.tar.gz (sig)

The development repository is also available via mercurial: http://hg.bikemonkey.org/javasrc_1_6_jrl_darwin/. The release tag is jdk6_devpreview_r2.

By downloading this source code, you certify that you are a Licensee in good standing under the Java Research License of the Java 2 SDK, and that your access, use, and distribution of code and information you may obtain at this site is subject to the License. Please review the license at http://java.net/jrl.csp, and submit your license acceptance to Sun.

Binary Access

I am making available the SoyLatte binary release. (Red Hat already claimed IcedTea, and I drank a lot of double soy lattes while working on this. Plus, I think it's funny).

SoyLatte is based on the BSD Port of Sun's Java 6 JDK, and is made available under the Java Research License. JDK and Java are trademarks of Sun. Like RedHat, I want to make it exceptionally clear that while SoyLatte is a port of Java, it is not Sun's Java, JDK, or OpenJDK. Unlike IcedTea, SoyLatte is made available under the JRL. Please see below for a licensing discussion.

By downloading these binaries, you certify that you are a Licensee in good standing under the Java Research License of the Java 2 SDK, and that your access, use, and distribution of code and information you may obtain at this site is subject to the License. Please review the license at http://java.net/jrl.csp, and submit your license acceptance to Sun.

Downloads

Binaries are available for Mac OS X 10.4 and 10.5 (32-bit), and Mac OS X 10.5-only (64-bit). The soylatte directory can be placed anywhere on your system -- I chose /usr/local/soylatte16-amd64. Like other Java platforms, setting the JAVA_HOME and PATH environmental variables to point at these locations will work as expected.

32-bit JDK for Mac OS X 10.4 and 10.5: soylatte16-i386-r2.tar.gz (sig)

64-bit JDK for Mac OS X 10.5: soylatte16-amd64-r2.tar.gz (sig)

To ensure compliance, downloading the source requires authentication:

Username: 'jrl' and Password: 'I am a Licensee in good standing'

Feature TODO List

I'll start looking at all of these eventually, but feel free to jump in:

JSR 223 JavaScript Support

Sun has pulled Rhino from the JRL and OpenJDK (GPLv2) sources, due to concerns regarding licensing. As a work-around, there is a BSD-licensed javascript JSR223 engine implementation, using Rhino, available via https://scripting.dev.java.net/:

We have built and tested with Rhino version 1.6 release 5. There is a Rhino based JavaScript engine bundled in JDK 6 (http://jdk6.dev.java.net). The JDK 6 bundled version is based on Rhino 1.6 release 2. Unlike JDK 6 bundled engine, all Rhino features (optimizer, E4X) are enabled in this version.

This seems to work as a drop-in replacement for Sun's sun.org.mozilla code. Here's JavaScript Hello World using Rhino 1.6r5 and the Scripting package:

java -cp ~/Downloads/jsr223/javascript/build/js-engine.jar:/tmp/rhino1_6R7/js.jar:. EvalScript
Found engine factory: com.sun.phobos.script.javascript.RhinoScriptEngineFactory
Found engine factory: com.sun.phobos.script.javascript.EmbeddedRhinoScriptEngineFactory
Hello, World

I am looking into the feasibility of integrating this solution.

Licensing

The Mac OS X work is based heavily on the BSD Java port, which is licensed under the JRL. The BSDs develop Java under the JRL; FreeBSD has negotiated a license with Sun to distribute FreeBSD Java binaries based on the JRL sources.

As the Mac port stabilizes, I am merging my work upstream into the BSD port, and in turn, it is a goal of the FreeBSD Java project to merge their work into OpenJDK. I've signed a Sun Contributor Agreement in preparation for this, and an OpenJDK Porters group has been proposed: http://thread.gmane.org/gmane.comp.java.openjdk.general/630

While the JRL makes this initial port possible, OpenJDK's GPLv2+CE licensing makes development and distribution far simpler. I hope to contribute this work to OpenJDK as soon as is feasible.

Submitting Bug Reports

There are bugs, and you're likely to find one. The most useful bug report is one that includes a simple, compilable reproduction case -- that gives me what I need to track down the really tricky bugs.

In submitting a bug, please include the following information:

Bug reports may be submitted to landonf (at) macports (dot) org.

[/code/macosx] — permanent link

Preview: Native Carbon SWT

21 Nov 2007, 17:13 PST

Native widgets via the Carbon SWT Implementation

The native Carbon SWT implementation appears to be working with the 32-bit Preview Release Mac OS JDK, without X11. This is the first step towards running SWT applications like Eclipse.

Non-X11 SwingWT Demo

The SwingWT project provides an implementation of most of the Swing API, using SWT to provide native graphics. While it's not a production solution, I thought it'd be fun to show a quick Hello, World demo with native Swing widgets, using SwingWT.

[/code/macosx] — permanent link

Open Source Java 6 on Leopard

19 Nov 2007, 22:56 PST

NOTE: You probably want to check out the latest release

Announcing Developer Preview Release 1

I'm pleased to announce the first Developer Preview Release of the open-source port of Java 6 to Mac OS X. This release includes support for 32-bit and 64-bit Intel machines running Mac OS X Leopard (10.5).

Port Status

(Nearly) everything up to and including Swing (X11) is functional. Sound is not currently supported.

While I've spent some time testing this release with my own projects, this preview release should be considered beta quality. The project is very much in need of additional community testing -- If you're hankering for Java 6, please give this a try! See below for information on submitting bug reports.

This work represents quite a few of my weekend and evening hours -- I hope you find it useful! Additionally, this port wouldn't be possible without building on the amazing FreeBSD, OpenBSD, and NetBSD porting work.

Tiger Support

While bugs in Tiger's compiler currently prevent the building of the 32-bit JDK, it should be possible to build the JDK on Leopard using MACOSX_DEPLOYMENT_TARGET and the MacOSX 10.4u SDK.

I plan on working on Tiger support over Thanksgiving. If you get this working before I get around to it, let me know!

Supporting PowerPC

Implementing PowerPC hotspot/interpreter support would require an incredible amount of work, and is not something I have planned. Most impressively, Gary Benson of Red Hat has implemented initial PowerPC interpreter support for OpenJDK. It's possible that this could be used to implement Mac OS X/PowerPC at some future date. I still have PowerPC Macs, so I'll look into this eventually.

Fetching the Sources

Sources are available as a downloadable archive, or from a mercurial repository. To download the source code, you must be a licensee in good standing under the Java Research License. To ensure compliance, downloading the source requires authentication:

Username: 'jrl' and Password: 'I am a Licensee in good standing'

Download: jdk6_devpreview_r1.tar.gz

The development repository is also available via mercurial: http://hg.bikemonkey.org/javasrc_1_6_jrl_darwin/. The release tag is jdk6_devpreview_r1.

By downloading this source code, you certify that you are a Licensee in good standing under the Java Research License of the Java 2 SDK, and that your access, use, and distribution of code and information you may obtain at this site is subject to the License. Please review the license at http://java.net/jrl.csp, and submit your license acceptance to Sun.

Building the JDK

Building the full JDK will take quite some time -- up to an hour on my Mac Pro. You can tune the number of parallel jobs with the HOTSPOT_BUILD_JOBS and PARALLEL_BUILD_JOBS make tunables (eg, PARALLEL_BUILD_JOBS=4), which will speed things up considerably on multi-core machines.

There are two builds available for the 64-bit and 32-bit VMs: debug, and release. The debug JRE is significantly slower than the release version -- for day to day work, I suggest using the release build.

OpenMotif is a required build dependency. I've installed it using MacPorts. If you install it using Fink, or by hand, be sure to update the ALT_MOTIF_DIR build setting accordingly.

When the build is complete, a full JDK image will be available in javasrc_1_6_jrl_darwin/control/build/<build style>/j2sdk-image. You can copy/rename the j2sdk-image directory as appropriate (ie, /usr/local/darwin-jdk16-32bit). To use the new JDK, set the JAVA_HOME environmental variable and (optionally) add the j2sdk-image directory to the start of your path.

Lastly, I expect the 64-bit JVM to be faster than the 32-bit JVM, but I've yet to do benchmarks.

64-bit Release Version:

cd control/make
env DYLD_LIBRARY_PATH=`pwd`/../build/bsd-amd64/lib/amd64/server \
make ALT_BOOTDIR=/System/Library/Frameworks/JavaVM.framework/Home \
ALT_MOTIF_DIR=/opt/local SYS_CFLAGS="" LANG="C" JAVA_HOME="" CLASSPATH="" \
LD_LIBRARY_PATH="" MAKEFLAGS="" SKIP_COMPARE_IMAGES="YES" \
BUILD_DEPLOY="false" ALT_DEVTOOLS_PATH=/usr ALT_CUPS_HEADERS_PATH=/usr/include \
HOTSPOT_BUILD_JOBS=1 PARALLEL_BUILD_JOBS=1 BSD_OVERRIDE_ARCH=amd64

The JDK will be built in control/build/bsd-amd64/j2sdk-image

64-bit Debug Version:

cd control/make
env DYLD_LIBRARY_PATH=`pwd`/../build/bsd-amd64-debug/lib/amd64/server \
make ALT_BOOTDIR=/System/Library/Frameworks/JavaVM.framework/Home \
ALT_MOTIF_DIR=/opt/local SYS_CFLAGS="" LANG="C" JAVA_HOME="" CLASSPATH="" \
LD_LIBRARY_PATH="" MAKEFLAGS="" SKIP_COMPARE_IMAGES="YES" \
BUILD_DEPLOY="false" ALT_DEVTOOLS_PATH=/usr ALT_CUPS_HEADERS_PATH=/usr/include \
HOTSPOT_BUILD_JOBS=1 PARALLEL_BUILD_JOBS=1 BSD_OVERRIDE_ARCH=amd64 debug_build

The JDK will be built in control/build/bsd-amd64-debug/j2sdk-image

32-bit Release Version:

cd control/make
env DYLD_LIBRARY_PATH=`pwd`/../build/bsd-i586/lib/i386/client \
make ALT_BOOTDIR=/System/Library/Frameworks/JavaVM.framework/Home \
ALT_MOTIF_DIR=/opt/local SYS_CFLAGS="" LANG="C" JAVA_HOME="" CLASSPATH="" \
LD_LIBRARY_PATH="" MAKEFLAGS="" SKIP_COMPARE_IMAGES="YES" \
BUILD_DEPLOY="false" ALT_DEVTOOLS_PATH=/usr ALT_CUPS_HEADERS_PATH=/usr/include \
HOTSPOT_BUILD_JOBS=1 PARALLEL_BUILD_JOBS=1

The JDK will be built in control/build/bsd-i586/j2sdk-image

32-bit Debug Version:

cd control/make
env DYLD_LIBRARY_PATH=`pwd`/../build/bsd-i586-debug/lib/i386/client \
make ALT_BOOTDIR=/System/Library/Frameworks/JavaVM.framework/Home \
ALT_MOTIF_DIR=/opt/local SYS_CFLAGS="" LANG="C" JAVA_HOME="" CLASSPATH="" \
LD_LIBRARY_PATH="" MAKEFLAGS="" SKIP_COMPARE_IMAGES="YES" \
BUILD_DEPLOY="false" ALT_DEVTOOLS_PATH=/usr ALT_CUPS_HEADERS_PATH=/usr/include \
HOTSPOT_BUILD_JOBS=1 PARALLEL_BUILD_JOBS=1 debug_build

The JDK will be built in control/build/bsd-i586-debug/j2sdk-image

Submitting Bug Reports

There are bugs, and you're likely to find one. The most useful bug report is one that includes a simple, compilable reproduction case -- that gives me what I need to track down the really tricky bugs.

In submitting a bug, please include the following information:

Bug reports may be submitted to landonf (at) macports (dot) org.

[/code/macosx] — permanent link

FreeBSD's 1.6 JDK on Mac OS X: Status Update

11 Nov 2007, 22:52 PST

Puzzle Pirates - A More Entertaining Hello World

Status Update

I've spent some more time working on porting JDK 1.6 -- adding support for Tiger (10.4), fixing stack alignment issues for both amd64 and i386, fixing build system issues, and more. On Leopard, the 32-bit JDK now builds to completion. As evidenced by the above "Hello, World", X11 Swing is working on the 32-bit JVM.

Work is still progressing on the 16-byte stack alignment issue. In the meantime, I've enabled the gcc -mstackrealign option to force stack alignment in compiled JVM code. This ensures that calls back into the JVM are properly aligned:

On the Intel x86, the -mstackrealign option will generate an alternate
prologue and epilogue that realigns the runtime stack. This supports mixing
legacy codes that keep a 4-byte aligned stack with modern codes that keep a
16-byte stack for SSE compatibility. The alternate prologue and epilogue are
slower and bigger than the regular ones, and the alternate prologue requires
an extra scratch register

To replace this temporary work-around, I'm currently working on fixing the call stack alignment in the HotSpot Interpreter. Currently most calls are properly aligned, with the JVM running until it hits a JNI call. The JNI code generation assumes that arguments can be pushed directly to the stack prior to calling the stub generator, making alignment more difficult. I have not started work on alignment in the c1 or c2 compilers. Due to -mstackrealign compiler bugs in Tiger, the JVM must be built on Leopard -- however, it should be possible to build a JDK to run on Tiger.

For 64-bit Leopard systems, I've fixed up some latent alignment issues in HotSpot (Leopard is very picky about checking alignment in dyld), and the code appears to work fine with a few caveats. I've only spent a few hours on this today, but 64-bit Java should be the most stable and easiest to port. There is a known issue in the AMD64 JVM that I have not had time to track down, occaisionally triggering the following assert:

 javasrc_1_6_jrl/hotspot/src/cpu/amd64/vm/frame_amd64.cpp:118
  assert((intptr_t) sp() <= (intptr_t) result,
           "result must >= than stack pointer");

As a temporary work-around, the x86_64 JVM appears to run fine (if slowly) in interpreter mode (-Xint).

Future Direction

As a FreeBSD-Java committer, I will be merging low-risk changes upstream to the FreeBSD Java Project (Code). Potentially destabilizing changes to Hotspot interpreter / compiler will not be merged until they've seen adequate review by the FreeBSD and OpenJDK teams.

There are tentative plans to merge the BSD Java changes into the OpenJDK project. This isn't something I'm directly involved with, but I have signed the Sun Contributor Agreement in preparation.

I am also very interested in external contribution -- this is a part time, after-work project for me! See below for code access.

In progress work:

Code Access

I am making the JRL-licensed source code available via a Mercurial repository. Since the OpenJDK project is moving to Mercurial, I thought I'd get on the bandwagon -- it is working out well. The repository is available at:

http://hg.bikemonkey.org/javasrc_1_6_jrl_darwin/

To download the source code, you must be a licensee in good standing under the Java Research License. To ensure compliance, the repository requires authentication -- the username is "jrl", and the password is 'I am a Licensee in good standing'. By downloading this source code, you certify that you are a Licensee in good standing under the Java Research License of the Java 2 SDK, and that your access, use, and distribution of code and information you may obtain at this site is subject to the License. Please review the license at http://java.net/jrl.csp, and submit your license acceptance to Sun.

Currently, all FreeBSD JDK16 work is under the JRL. In the future, the work may be relicensed, with a copyright grant, for OpenJDK under the GPLv2.

This not something I like to do, but I request that all submissions assign copyright to myself, such that I have the ability to relicense/grant copyright for submission to the upstream projects. If anyone has a better idea of how to handle this, I'm very interested.

To build the x86_32 JDK:

cd control/make
env DYLD_LIBRARY_PATH=`pwd`/../build/bsd-i586-debug/lib/i386/client \
make ALT_BOOTDIR=/System/Library/Frameworks/JavaVM.framework/Home \
ALT_MOTIF_DIR=/opt/local SYS_CFLAGS="" LANG="C" JAVA_HOME="" CLASSPATH="" \
LD_LIBRARY_PATH="" MAKEFLAGS="" SKIP_COMPARE_IMAGES="YES" \
BUILD_DEPLOY="false" ALT_DEVTOOLS_PATH=/usr ALT_CUPS_HEADERS_PATH=/usr/include \
HOTSPOT_BUILD_JOBS=1 PARALLEL_BUILD_JOBS=1 debug_build

To build the x86_64 (amd64) JDK:

cd control/make
env DYLD_LIBRARY_PATH=`pwd`/../build/bsd-amd64-debug/lib/amd64/server \
make ALT_BOOTDIR=/System/Library/Frameworks/JavaVM.framework/Home \
ALT_MOTIF_DIR=/opt/local SYS_CFLAGS="" LANG="C" JAVA_HOME="" CLASSPATH="" \
LD_LIBRARY_PATH="" MAKEFLAGS="" SKIP_COMPARE_IMAGES="YES" \
BUILD_DEPLOY="false" ALT_DEVTOOLS_PATH=/usr ALT_CUPS_HEADERS_PATH=/usr/include \
HOTSPOT_BUILD_JOBS=4 PARALLEL_BUILD_JOBS=4 BSD_OVERRIDE_ARCH=amd64 JAVA_JVM_FLAGS=-Xint JAVAC_FLAGS=-Xint debug_build

Remove the "-Xint" flags to disable interpreter mode (and trigger the known issue in frame_amd64.cpp).

Note that the debug build is considerably slower than a production build, and interpreted mode is even slower!

If you're interested in assisting with IA32 stack alignment issues, the current patch is available here: patch-stack-align-darwin-v2.

Looking for a Job?

Lastly, a plug for my group at Three Rings. We're looking to hire Java Engineers (not just billing!) and FreeBSD Developer/Admins. The group rocks (I run it!), so feel free to drop me a line if you're interested.

[/code/macosx] — permanent link

FreeBSD's 1.6 JDK on Mac OS X

04 Nov 2007, 21:52 PST

NOTE: You probably want to check out the latest updates

Introduction

I've long wondered what it would take to get the FreeBSD Java Port running on OS X, so this weekend I spent a couple days getting Java 1.6 running on my x86 Leopard machine.

Weekend is over, and I can report partial success -- hotspot compiles, the jre mostly bootstraps, and Hello World runs. Anything complex appears to trigger stack alignment issues (Apple's i386 API requires a 16-byte aligned stack)

landonf@max:javasrc_1_6_jrl/control/build/bsd-i586> ./bin/java -version
java version "1.6.0_03-p3"
Java(TM) SE Runtime Environment (build 1.6.0_03-p3-landonf_04_nov_2007_00_06-b00)
Java HotSpot(TM) Server VM (build 1.6.0_03-p3-landonf_04_nov_2007_01_30-b00, mixed mode)
landonf@max:javasrc_1_6_jrl/control/build/bsd-i586> ./bin/java Hello
Hello World!

What's missing?

Missing pieces:

Additionally, there's no Aqua support -- Swing would require X11.

Source? Compiling?

The patch is based on the BSD 1.6 JDK patchset, which means you need to download Sun's JDK source, the FreeBSD patchset, AND the Darwin patches. Instructions on downloading the Sun source, and the FreeBSD patchset, are available from the FreeBSD JDK site

The Darwin patchset should be applied on top of the BSD patchset. By downloading this source code, you certify that you are a Licensee in good standing under the Java Research License ("License") of the Java(tm) 2 SDK, and that your access, use and distribution of code and information you may obtain at this site is subject to the License. Download patch-jdk16-apple-alpha1.

Compilation requires OpenMotif, Nawk, and possiblity additional software -- I installed all the dependencies using MacPorts. To build, cd to control/make and run the following (tweaked appropriately):

env DYLD_BIND_AT_LAUNCH=1 \
DYLD_LIBRARY_PATH=<javasrcpath>/control/build/bsd-i586/lib/i386/client \
make ALT_BOOTDIR=/System/Library/Frameworks/JavaVM.framework/Home \
ALT_MOTIF_DIR=/opt/local SYS_CFLAGS="" LANG="C" JAVA_HOME="" CLASSPATH="" \
LD_LIBRARY_PATH="" MAKEFLAGS="" SKIP_COMPARE_IMAGES="YES" \
DONT_BUILD_DEPLOY="YES" ALT_DEVTOOLS_PATH=/usr ALT_CUPS_HEADERS_PATH=/usr/include \
HOTSPOT_BUILD_JOBS=4 PARALLEL_BUILD_JOBS=4

The build does not complete (yet), but a bootstrap JVM is built (and can be run from) ./control/build/bsd-i586/bin/java

[/code/macosx] — permanent link