Tackling the OpenJDK can be intimidating.Therefore, let's....
figure out what's in the OpenJDK
discover how to build
examine the layout of the OpenJDK sources
make a trivial(!) change and build
My goal is not to teach you all of the OpenJDK
no way I can do that in just one lecture
My goal is to teach you enough that you can start learning
in other words, get you started, then shove you out of the nest
In order to make the most of this talk, you should...
be comfortable with C++
really really helps to be comfortable with JNI
be (very very!) comfortable with Java
understand that this is a large (millions of lines) codebase
large codebases do things a little differently
have a fast machine with lots of RAM, disk space, and a fat pipe
In the beginning...
Java was a closed-source, proprietary system
sort of
Back in the earliest days of Java (JDK 1.0.2)
you could download an unofficial source build
but it was
unsupported
unbuildable (for the most part)
unpatchable
in a nutshell, it wasn't officially "open source"This held true for all versions of Java before Java5
This non-FOSS arrangement was not popular
several Java alternatives emerged:
Project Classpath
IcedTea
... and a few other coffee puns
IBM started an OSS JVM: Harmony
several, actually, including a JVM-written-in-Java (Jikes RVM)
a few others explored the idea
Note that the sources were available, just not free
Sun sold source licenses of the JDK code
Microsoft was one such licensee
Sun maintained a distinction between "spec" and "implementation"
Java5: Time to go open-source!
Java5: Time to go open-source!
much, much easier said than done
required total modernization of the tools
particularly for Windows builds
required storage in OSS repositories
they chose Mercurial
required acceptance of outside patches
which they didn't, at first
required FOSS-equivalents to non-FOSS components
TrueType fonts, video and audio codecs, and so on
Java6: Almost there
by the time Java6 shipped, Sun had almost gotten everything done
Windows build tooling were mostly up-to-date
outside commitments slowly being integrated
but problems still remained
namely, all those non-FOSS equivalents
so "OpenJDK" became another JVM implementation
as opposed to "Sun"/"Oracle" JDK
This bifurcation remains true today
OpenJDK: completely open-source
OracleJDK: technically commercial and closed-source
but derives from OpenJDK source base
... and this will probably remain true forever
or at least as long as those non-FOSS components are in use
OpenJDK is a mix of Java and C++ source
much of the code is Java
various java.*, javax.* packages, some of which have native methods
"core" native code bits are Hotspot
garbage collectors, JIT, threading, and so on
"servicability" bits are mixed native/Java
JMX, instrumentation, etc
"langtools" (compilers, etc) are almost all Java
Release pages:** JDK 6--- http://openjdk.java.net/projects/jdk6/** JDK 7--- http://openjdk.java.net/projects/jdk7/** JDK 8--- http://openjdk.java.net/projects/jdk8/
Note that JDK 'updates' are a different project!** for gnarly legal and licensing reasons** JDK 7u--- updates to the JDK 7 project** JDK 8u--- updates to the JDK 8 project** all development has closed down on the non-update repositories!--- I am going to assume you don't care about pre-8u sources
Building OpenJDK 8u** Clone the root repo--- hg clone http://hg.openjdk.java.net/jdk8u/jdk8u** Pull the "dependent" source trees--- cd jdk8u; bash get_source.sh
Source layout** a "root project" with several "subprojects"** technically this is a Mercurial "forest"--- but that's really complicated, so they simplified by adding the get_source script** subprojects:*** corba*** hotspot*** jaxp*** jaxws*** jdk*** langtools*** nashorn
Source layout** within each subproject, usually a "make" and "src"--- sometimes a "test" directory** "make" typically is black-box build stuff** "src" usually breaks down into*** platform-specific ("windows", "share")*** Java- vs native ("classes" vs not "classes")*** and for "classes", the usual package-qualified breakdown
Configure the makefiles** run the "configure" script to configure all the makefiles--- bash configure** note that any missing dependencies will be flagged and configure will terminate** check out ./README-builds.html for build details
Requirements** All:*** a "boot JDK" installation for Java files compilation*** generally the boot JDK must be one version behind (JDK 7)** MacOS:--- Xcode 4 (!) and command-line tools** Windows:*** Visual Studio 2013*** Cygwin (for Unix-style build chain)** Linux:--- usual build chain (g++, etc)
Release page:** JDK 9--- http://openjdk.java.net/projects/jdk9/** big change:--- complete project restructure for modules
Building OpenJDK 9:** Clone root repo:--- hg clone http://hg.openjdk.java.net/jdk9/jdk9** Pull the "dependent" source trees--- cd jdk9; bash get_source.sh
Source layout** a "root project" with several "subprojects"** technically this is a Mercurial "forest"--- but that's really complicated, so they simplified by adding the get_source script** subprojects:*** corba*** hotspot*** jaxp*** jaxws*** jdk*** langtools*** nashorn
Source layout** within each subproject, usually a "make" and "src"--- sometimes a "test" directory** "make" typically is black-box build stuff** "src" usually breaks down into*** java modules ("java.desktop", "java.rmi", etc)*** or os-specific ("bsd", "solaris", "windows", etc)** within each module, breaks down into*** "share" (os-independent) or os-specific*** "classes" (Java) v "native" (C++)
Configure the makefiles** run the "configure" script to configure all the makefiles--- bash configure** note that any missing dependencies will be flagged and configure will terminate** check out ./README-builds.html for build details
Requirements** All:*** a "boot JDK" installation for Java files compilation*** generally the boot JDK must be one version behind (JDK 7)** MacOS:--- XCode and command-line tools** Windows:*** Visual Studio 2015*** Cygwin (for Unix-style build chain)** Linux:--- usual build chain (g++, etc)
Release pages:
JDK 10-- http://openjdk.java.net/projects/jdk/10/
JDK 11
http://openjdk.java.net/projects/jdk/11/
JDK 12
http://openjdk.java.net/projects/jdk/12/
... and so on
Github repositories:
https://github.com/openjdk/...
jdk: The current development path
jdkn: The (past) development for JDK n (these are often archived)
jdknu: For rampdown of the updated releases of JDK n
jdknu-dev: The updates/errata/bugfixes for JDK n
Building OpenJDK project
Clone root repo:
Go get JDK 10, use branch name (described in release page)
Note that all future JDK releases will use timestamped version name
18.3 (March 2018) or 18.9 (Sept 2018)
Source layout
one large project, made up of modules
did away with "subprojects"
each module is made up of Java and native source
Source layout
demo, sample
solaris, bsd, linux
hotspot, utils
java.base, java.compiler, java.se, ...
Source layout
each module broken into parts
share
os (linux, windows, bsd, etc)
nashorn
within that, further breakdown
classes: Java code
native: C/C++ code
conf: configuration files (usually just copied)
lib: external libraries
legal: legal disclaimers/messages/licenses
doc: man-page documentation
Configure the makefiles
run the "configure" script to configure all the makefiles
bash configure
note that any missing dependencies will be flagged and configure will terminate
check out ./README-builds.html for build details
Requirements
All:
a "boot JDK" installation for Java files compilation
generally the boot JDK must be one version behind (JDK 7)
MacOS:
XCode 9 and command-line tools
Windows:
Visual Studio 2017
Cygwin (for Unix-style build chain)
Linux:
usual build chain (g++, etc)
Configure flags of note
run configure --help for full list
three different kinds of builds:
release
optimized (release + some debug flags)
fastdebug (debug w/some optimizations)
slowdebug (debug w/o optimizations)
Find the java "launcher" code (java / java.exe)
Change the java.vendor system property
Get a list of all the -XX flags in a diagnostic build
Find where -XX:DisableExplicitGC is checked
Write your own garbage collector
make sure to handle any size heap from 8m to 1024GB
make sure to do any GC pass on O(1) time
Who is this guy?
Architect, Engineering Manager/Leader, "force multiplier"
Principal -- Neward & Associates
http://www.newardassociates.com
Educative (http://educative.io) Author
Performance Management for Engineering Managers
Author
Professional F# 2.0 (w/Erickson, et al; Wrox, 2010)
Effective Enterprise Java (Addison-Wesley, 2004)
SSCLI Essentials (w/Stutz, et al; OReilly, 2003)
Server-Based Java Programming (Manning, 2000)