Recommended: Sing it, brah! 5 fabulous songs for developers
JW's Top 5
OpenJDK, the open-source JDK 7 release (and no, I don't know if there's any practical
difference between the two) has officially opened for business with the promotion
of the "real,
live" Mercurial repositories. These are the real deal, the same repositories that
Sun employees will be working on as they modify the code... which means, in all reality,
that there is a very tiny window of opportunity for you to check out code
between changesets that are dependent on one another due to the way they've got the
forest set up--if you get weird build errors, try re-fetching... but more on that
later.
Think about it for a second--now, thanks to the way Mercurial handles source (a distributed
source code system is different from a centralized client/server one like SVN or CVS),
you can hack away on your own private build of the JDK, and still keep up to date
with what Sun's doing. And, if Sun likes what you're doing, you could end up contributing
back to the JDK as a whole.
You could make changes to the langtools bundle to start adding new features you think
should be in Java.
You could make changes to the hotspot bundle to start exploring ways to optimize GC
better.
You could fix bugs you find in the various Java libraries.
You could even add new features to the core classes that you've been saying needed
to be there since Day 1.
You can start exploring the new Modules system (JSR-277) strawman implementation.
Then, you can post your diffs, or just tell people where your changesets are, and
others can play with them.
Cool.
Meanwhile, for those of you looking to get started with hacking on the JDK, there's
a couple of things you need to do:
Naturally, things are just a teeny bit more complicated than that, particularly
if you're on a Windows box. (Update: Volker Simonis has blogged,
similar to this post, his
experience in configuring a Suse Enterprise Linux 9.3 box to build the OpenJDK.)
Thus, as one who's labored through this process on said platform, I'll show you how
I got it all up and running on my machine (well, technically, pseudo-machine, it's
a VMWare image with Windows XP SP2 in it). If you have questions, you're free to send
me email, but chances are I'll just redirect you to build-dev, where the amazing Kelly
O'Hair hangs out and answers build questions. (MAJOR kudos again to Kelly for getting
this done!)
Note that Sun is looking at other platforms, and has some good docs on building for
Ubuntu 6.x and 7.x in the README already known--I've managed that with very little
difficulty, so if you're starting from scratch, I'd suggest grabbing the free VMWare
Player, an Ubuntu 7.04 Desktop image, and use that as a JDK build environment.
While we're at it, along the way, I'll be pointing out areas that I think Sun could
use and would appreciate some community contributions. (People are always asking me
this at conferences, as a way of making their name more well-known and building credibility
in the industry.) Note that I have no "pull" with Sun and can't begin to guess where
they want the help, I'm simply suggesting where I think the help would be
useful--to the community if not to Sun directly.
By the way, the official document for build process requirements is the OpenJDK
Build README; I only offer this up as a supplement and to offer up my own experiences.
Note that I didn't go "all the way", in that I don't care about building the Java
Plug-In or some of the other ancillary stuff, such as the JDK installer itself, so
I didn't necessarily install all the bits documented on the OpenJDK README page. I
wanted source that I could hack, step into, debug, and so on.
Without further ado....
This sort of goes without saying, but in order to build the JDK, you need to have
the necessary tools in place.
I simply cloned the forest repository in
c:\Mercurial.
In a cygwin terminal:cd c:/Mercurial hg clone http://www.terminus.org/hg/hgforest hgforestThen I edited
c:/Mercurial/Mercurial.iniand added the lines:[extensions] forest=c:/Mercurial/hgforest/forest.pyas documented in the Mercurial
Wiki.
At this point, you should be ready to go.
Ivan's
got it (mostly) right: just do this:
cd C:\Prg\OpenJDK
md jdk7
hg fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7
(Don't use MASTER as he does, I don't think that works--that was just for the experimental
repositories.) Don't forget the trailing slash, either, or you'll get an error saying
something along the lines of http://hg.openjdk.java.net/jdk7%5Ccorba is not a valid
URL or somesuch.
If your Mercurial doesn't have the "forest" extension, "fclone" won't work; Ivan's
got tips on how to pull down the sub-repositories by hand, but I get nervous doing
that because what if the list changes and I wasn't paying attention?
Your network will go wild for about twenty minutes or so, maybe longer, pulling lots
of stuff down from the URL above. The sources should now reside in C:\Prg\OpenJDK\jdk7.
Go browse 'em for a bit, if you feel so inclined. Get a good rush going, because this
next step can be the tricky one.
Update: Volker Simonis ran into some issues with using Mercurial
and an HTTP proxy, and found it difficult to find assistance across the Web. In the
interests of making it easier for others, he's allowed me to republish his experience
here:
I just had a real hard time to get the forest extension working and finally found
out that it was because the forest extension doesn't honor the "http_proxy" environment
variable. So I thought I'll post it here in case anybody else will face the same problem
in order to save him some time. (If you're only interested in the solution of the
problem, you can skip the next paragraphs and jump right to the end of this post).I installed Mercurial and the forest extension as described in various places, here
on the list and on the Web - that was the easy part:) Afterwards I could do:/share/software/OpenJDK> hg clone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7 requesting all changes adding changesets adding manifests adding file changes added 2 changesets with 26 changes to 26 files
26 files updated, 0 files merged, 0 files removed, 0 files unresolvedSo everything worked fine! Note that I'm behind a firewall, but Mercurial correctly
picked up my http proxy from the "http_proxy" environment variable!But now, everytime I tried 'fclone', I got the following error:
/share/software/OpenJDK> hg fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7 [.] abort: error: Name or service not knownThat was not amusing. First I thought I got something wrong during the installation
of the forest extension. I than used the '--traceback' option to "hg" which told me
that the error was in keepalive.py:/share/software/OpenJDK> hg --traceback fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7 [.] Traceback (most recent call last): ... File "/share/software/Python-2.5.1_bin/lib/python2.5/site-packages/mercurial/keepalive.py", line 328, in _start_transaction raise urllib2.URLError(err) URLError: <urlopen error (-2, 'Name or service not known')> abort: error: Name or service not knownSo I enabled the debugging output in keepalive.py and realized, that the first two
connections to hg.openjdk.java.net where made trough the proxy, while the third one
(the first that actually fetches files), wants to go directly to hg.openjdk.java.net,
which badly fails:/share/software/OpenJDK> hg fclone http://hg.openjdk.java.net/jdk7/jdk7/ jdk7 keepalive.py - creating new connection to proxy:8080 (1078835788) keepalive.py - STATUS: 200, OK keepalive.py - re-using connection to proxy:8080 (1078835788) keepalive.py - STATUS: 200, OK [.] keepalive.py - creating new connection to hg.openjdk.java.net (1078970092) abort: error: Name or service not knownThe problem can be fixed by adding the proxy settings to your .hgrc file, like this:
[http_proxy] host=proxy:8080where you have to replace "proxy:8080" with the name and the port of your real proxy
host!
Volker's original email came from the buid-dev list, and if you have further questions
about Mercrial and HTTP proxies, I'd suggest that as a resource.
This is where you'll spend a fair amount of time, because getting this right can be
ugly. There's some environment variables that tell the build script where to find
things, and we have to point out those things, like compiler location and such. If
you installed everything in the same places I did, then the following, which I put
into C:\Prg\OpenJDK\build_shell.sh, should work for you:
#!/bin/sh
# "External" bits (outside of OpenJDK path structure)
#
export ALT_BOOTDIR=C:/Prg/jdk1.6.0
export ANT_HOME=c:/Prg/apache-ant-1.7.0
export FINDBUGS_HOME=/cygdrive/c/Prg/findbugs-1.2.1# OpenJDK flag (to make FreeType check pass)
#
export OPENJDK=trueexport OPENJDK_HOME=C:/Prg/OpenJDK
openjdkpath=$(cygpath --unix $OPENJDK_HOME)# OpenJDK-related bits
# (ALT_JDK_IMPORT_PATH fixes a corba bug; remove it later)
#
export ALT_JDK_IMPORT_PATH=$(cygpath --unix C:/Prg/jdk1.6.0)
export ALT_BINARY_PLUGS_PATH=$OPENJDK_HOME/openjdk-binary-plugs
export ALT_UNICOWS_DLL_PATH=$OPENJDK_HOME/deps/unicows
export ALT_FREETYPE_LIB_PATH=$OPENJDK_HOME/deps/freetype-igor/windows/freetype-i586/lib
export ALT_FREETYPE_HEADERS_PATH=$OPENJDK_HOME/deps/freetype-igor/windows/freetype-i586/include/freetype2
. $openjdkpath/jdk7/jdk/make/jdk_generic_profile.sh# Need GNU make in front of Cygwin's; this is the only practical
way to do it
#
PATH=$openjdkpath/bin:$PATH
export PATH# Let people know this is an OpenJDK-savvy prompt
#
export PS1='OpenJDK:\[\e]0;\w\a\]\[\e[32m\]\u@${COMPUTERNAME}:\[\e[33m\]\w\[\e[0m\]\n\$
'
Note the UNICOWS_DLL thing in the middle; this was necessary in earlier builds, (Update: Kelly
and I don't know if it still is. For now I'm not messing with it; if you discover
that you need it, the Build README has links.
confirmed that this is no longer necessary in the OpenJDK build. Yay!)
Note that I set the COMPILER_VERSION flag to tell the build script which compiler
I'm using--if that's not set, the build fails pretty quickly, complaining that "COMPILER_VERSION"
cannot be empty. (Update: Kelly mentions, "I suspect the reason you
are having the COMPILER_VERSION problem is that the makefiles are trying to run cl.exe
to get the version, and if the PATH/LIB/INCLUDE env variables are not setup right,
cl.exe fails. Several people have run into that problem.")
Note that OPENJDK must be set, or the build process thinks this is a commercial build,
and an early sanity-check to see what version of FreeType is running will fail. (Specfically,
Sun builds a tool just to see if the code compiles; if it fails to compile, chances
are you forgot to set this flag. That's been my problem, each and every time I tried
to rebuild the OpenJDK build space. Hopefully I never forget it again.)
Note that I call into jdk_generic_profile.sh to do some more setup work; this gets
all the MSVS2003 stuff into the PATH as well.
Be very careful with which path you use; sometimes the build wants C:/Prg style paths,
and sometimes it wants /cygdrive/c/Prg style paths. Don't assume the script above
is perfect--I'm still testing it, and will update this entry as necessary as I find
bugs in it.
(Update: Kelly mentions, "Be careful putting cygwin before VS2003
in the PATH, cygwin has a link.exe and so does VS2003, you need the one from VS2003.")
From a Cygwin bash prompt,
cd /cygdrive/c/Prg/OpenJDK
. ./build_shell.sh
cd jdk7
make sanity
It will churn, think, text will go flying by, and you will (hopefully) see "Sanity
check passed". If not, look at the (voluminous) output to figure out what paths are
wrong, and correct them. Note that certain paths may be reported as warnings and yet
the buld will still succeed, that's OK, as far as I can tell.
And no, I don't know what all of those environment variables are for. Kelly might,
but I suspect there's a lot of built-up cruft from over the years that they'd like
to start paring down. Let's hope.
Specifically, type "make help" to see the list of targets.
OpenJDK:Ted@XPJAVA:/cygdrive/c/Prg/OpenJDK/jdk7
$ make help
Makefile for the JDK builds (all the JDK).--- Common Targets ---
all -- build the core JDK (default target)
help -- Print out help information
check -- Check make variable values for correctness
sanity -- Perform detailed sanity checks on system and settings
fastdebug_build -- build the core JDK in 'fastdebug' mode (-g -O)
debug_build -- build the core JDK in 'debug' mode (-g)
clean -- remove all built and imported files
clobber -- same as clean--- Common Variables ---
ALT_OUTPUTDIR - Output directory
OUTPUTDIR=./build/windows-i586
ALT_PARALLEL_COMPILE_JOBS - Solaris/Linux parallel compile run count
PARALLEL_COMPILE_JOBS=2
ALT_SLASH_JAVA - Root of all build tools, e.g. /java or J:
SLASH_JAVA=J:
ALT_BOOTDIR - JDK used to boot the build
BOOTDIR=C:/Prg/jdk1.6.0
ALT_JDK_IMPORT_PATH - JDK used to import components of the build
JDK_IMPORT_PATH=c:/Prg/JDK16~1.0
ALT_COMPILER_PATH - Compiler install directory
COMPILER_PATH=C:/Prg/MSVS2003/Common7/Tools/../../Vc7/Bin/
ALT_CACERTS_FILE - Location of certificates file
CACERTS_FILE=/lib/security/cacerts
ALT_DEVTOOLS_PATH - Directory containing zip and gnumake
DEVTOOLS_PATH=/usr/bin/
ALT_DXSDK_PATH - Root directory of DirectX SDK
DXSDK_PATH=C:/Prg/DIRECT~1
ALT_MSDEVTOOLS_PATH - Root directory of VC++ tools (e.g. rc.exe)
MSDEVTOOLS_PATH=C:/Prg/MSVS2003/Common7/Tools/../../Vc7/Bin/
ALT_MSVCRT_DLL_PATH - Directory containing mscvrt.dll
MSVCRT_DLL_PATH=C:/WINDOWS/system32
WARNING: SLASH_JAVA does not exist, try make sanity
WARNING: CACERTS_FILE does not exist, try make sanity--- Notes ---
- All builds use same output directory unless overridden with
ALT_OUTPUTDIR=<dir>, changing from product to fastdebug you may want
to use the clean target first.
- JDK_IMPORT_PATH must refer to a compatible build, not all past promoted
builds or previous release JDK builds will work.
- The fastest builds have been when the sources and the BOOTDIR are on
local disk.--- Examples ---
make fastdebug_build
make ALT_OUTPUTDIR=/tmp/foobar all
make ALT_OUTPUTDIR=/tmp/foobar fastdebug_build
make ALT_OUTPUTDIR=/tmp/foobar all
make ALT_BOOTDIR=/opt/java/jdk1.5.0
make ALT_JDK_IMPORT_PATH=/opt/java/jdk1.6.0OpenJDK:Ted@XPJAVA:/cygdrive/c/Prg/OpenJDK/jdk7
$
The one I want is "make fastdebug_build" or "make debug_build" (so I have debug symbols
and can go spelunking). Do it.
Watch the stuff go flying by.
Get bored.
Get lunch.
Come back, it might be done. :-)
If it's a successful build, you'll have "stuff" in the C:\Prg\OpenJDK\jdk7\build directory,
corresponding to the kind of build you kicked off; for a "fastdebug" build, for example,
there'll be a "windows-i586-fastdebug" directory in which you'll find a "j2sdk-image"
directory in which there should be a complete JDK environment. Try running
Java and see if it works (make sure your PATH is updated to point to the right place
before you do!)
If not, it's debugging time. :-) Note that the "build" directory is completely built
from scratch, so if you get a partial build and want to start over from scratch, just
"rd /s build" from the jdk7 directory. (It's easier than "make clean" or "make clobber",
I've found.)
When building the JDK, you may want to build bits "underneath" the top-level directory,
but doing this is a tad tricky. I asked about this on the build-dev list, and Kelly
responded with a great email about the build process, specifically about launching
"sub" makes within the system:
Due to history, a build directly from the jdk/make directories uses a default OUTPUTDIR
of jdk/build/* but if FASTDEBUG=true, it's jdk/build/*-fastdebug, or if a plain debug
build with just VARIANT=DBG it would be jdk/build/*-debug The variant builds leave
results in a completely separate outputdir.If you used the very top level makefile (which came from the now defunct control/make
area) the default OUTPUTDIR is ./build/* (at the very top of the repositories). When
this top level Makefile runs the jdk/make Makefiles, it passes in a ALT_OUTPUTDIR
to refer to this top level build result area because it's default outputdir is not
the same place.I don't know the complete history as to why this was done this way, but my tendency
is to try and get us back to a single default OUTPUTDIR for all the repositories.
Someday...This is what I do when I work on just the jdk repository:
cd jdk/make && gnumakeThat primes the outputdir area, then I can drop down in:
cd jdk/make/java && gnumakeOr even drop in and clean an area and re-build it:
cd jdk/make/jpda && gnumake clean && gnumakeOr just repeat the entire build (incremental build)
cd jdk/make && gnumakeIf I wanted the jdk image (j2sdk-image), I would need to:
cd jdk/make && gnumake imageBut the output by default will go to jdk/build/* and a different directory if VARIANT=DBG
or FASTDEBUG=true.
This should help having to go through the whole process for incremental updates. (Note
that on my system, I had to call it "make" instead of "gnumake".)
As time goes on, I'll hopefully find the time to blog about how to find various little
things in the JDK and make source-modifications to prove that they work, and use that
as a springboard from which to let you start hacking on the JDK. In the meantime, please,
if you run into trouble and find fixes to any of the above, comment or email me, and
I'll correct this.