Typo

[linux_from_scratch_hints.git] / OLD / javafromscratch.txt

TITLE:          Compiling Java 2 SDK from Scratch

LFS VERSION:    LFS-4.0

AUTHOR:         Tushar Teredesai <Tushar@LinuxFromScratch.Org>

SYNOPSIS:
        How to compile Java 2 SDK Source (available from Sun) using gcc3 so that
        you can use it as a plugin for your shiny new gcc3 compiled mozilla.

HINT:

Primary Location of this hint:
        http://www.linuxfromscratch.org/~tushar/
The latest version of the hint and any relevant patches are available at that
site. Please refer to the primary location before submitting
bug-reports/enhancements to this hint.

You may freely copy this document or create derivate works or distribute the
document in any format. At your discretion, you may give credit to the original
author:)

Use the hint at your own risk. Neither the author, nor the Linux From Scratch
project accepts any reponsibility for anything that happens when using these
documents or associated files.

An appropriate place to discuss this hint is blfs-support MailingList/NewsGroup
at LinuxFromScratch.Org.

Change Log:
[2002-01-13]
        * Updated to j2sdk-1.4.1.
        * Added fix so that system installed motif libraries are not linked.
[2003-01-03]
        * Added fix for lesstif to avoid conflict with gcj headers.
[2002-12-25]
        * Changed primary location and e-mail address.
[2002-11-13]
        * Removed -O3 from the Makefiles since it causes miscompilation.
[2002-10-27]
        * Upgraded the included zlib 1.1.3 to 1.1.4.
[2002-10-24]
        * Added compiling against system installed dynamic lesstif.
        * Added warning about OpenMotif in the workaround sections.
        * Fixed typo in libstdc++ version.
[2002-09-26]
        * Added info on making make verbose.
        * Added note on CFLAGS.
        * Added note for insanity mode.
[2002-08-31]
        * Added discussion on renaming i386 directories to i?86.
[2002-07-24]
        * Added link to German translation.
[2002-07-19]
        * Added CLASSPATH mini-hint.
[2002-07-18]
        * Optmizing the build.
        * Added Problems and Workarounds section.
        * Added fix by Harald Meier.
[2002-06-20]
        * First public version.

NOTE:
This hint and the attached patch is only useful for people who have accepted SUN
COMMUNITY SOURCE LICENSE for the Java 2 Platform Standard Edition. Note that you
are not allowed to distribute the resulting JDK/JRE and you also cannot use the
resulting JDK to compile programs for distribution unless they pass Sun's
compatiblity tests. In short don't use the resulting JDK except for your
personal use.

Future Plans:
* Use system installed Mozilla headers and build only Mozilla-1.0 plugin and
  Netscape Plugin. [Check the unfinished patch.]
* Remove the dependency on the sun's binary jdk and to use some source based
  compiler such as jikes/japhar/kaffe for bootstrapping.

Translations:
        * German by Jens Gutzeit
                <http://lfsforum.org/howtos/read.php?howto=7>

Pre-requisites (post LFS):
        * XFree86
                <http://freshmeat.net/projects/xfree86/>
        * InfoZip Zip and Unzip
                <http://freshmeat.net/projects/zip/>
        * InfoZip Unzip
                <http://freshmeat.net/projects/unzip/>
        * GNU cpio
                <http://freshmeat.net/projects/cpio>
        * Tenex C Shell
                <http://freshmeat.net/projects/tcsh/>
        * Sun JDK 1.4
                <http://freshmeat.net/projects/sunjdk/>
Installation of all these packages is covered in the BLFS book at
<http://beyond.linuxfromscratch.org>.

The JDK has a circular dependency and hence requires a Java compiler to compile
some parts for bootstrapping. You can install the JDK in a non-standard location
for compiling the SDK and then delete it. Also, it needs a 1.4 version JDK
according to Sun's website. If you don't want to install the jdk permenantly, I
have included instructions on how to install it for the duration of the
compilation. Using the same instructions, you can install the precompiled JDK
available from Sun.

Note that when I build the SDK, I had many packages already installed, so I
don't have the exact dependencies. If anyone runs into any problems regarding
missing dependencies, let me know and I will update this section.

Packages to download:
        * Zlib 1.1.4
                <http://freshmeat.net/projects/zlib>
Though this library may already be installed, you will need the source code to
update the source code that comes with j2sdk.
        * Java 2 SDK Source <http://wwws.sun.com/software/java2/download.html>
From Sun's website, download the following files (you will need to sign up and
agree to SCSL):
        - j2sdk-1_4_1-src-scsl-a.zip
        - j2sdk-1_4_1-src-scsl-b.zip
        - j2sdk-1_4_1-mozilla_headers-unix.zip
The last zip file may be referred to as Mozilla binaries at the above URL.
You will need to register at the website and agree to the "SUN COMMUNITY SOURCE
LICENSE" before downloading the code. Note that the file
j2sdk-sec-1_4_1-src-scsl.zip is not needed (unless you have a certificate from a
certificate authority).

Also download the patches from the primary site of this hint. Each patch should
have a short blurb on what it does.
You agree that by downloading the patch, you accept the SUN COMMUNITY SOURCE
LICENSE:)

This hint, unlike the mozilla and openoffice hints, is short and sweet:)


Installing the Binary Version for Bootstrapping:

You need an existing JDK Version 1.4 to bootstrap the build. You can use the
instructions below to install the jdk either in the current directory or in a
system wide directory (e.g. /opt/java/j2sdk-precompiled-1.4).
        VERSION=1.4.1_01
        V=`echo ${VERSION} | sed -e "s/\./_/g"`
        # Check out where the tar file begins and extract it
        T=`grep -a tail j2sdk-${V}-linux-i?86.bin | cut -f 2 -d " "`
        tail $T j2sdk-${V}-linux-i?86.bin > install.sfx.$$
        chmod +x install.sfx.$$
        # Extract the files and move them to where we want the jdk installed.
        ./install.sfx.$$
        cd j2sdk${VERSION}
        install -d /opt/java/j2sdk-precompiled-1.4.1
        mv * /opt/java/j2sdk-precompiled-1.4.1

This location is referred to as JAVA_HOME. So set the variable now.
        export JAVA_HOME=/opt/java/j2sdk-precompiled-1.4.1
Sun's binary JDK is linked against gcc2 libraries, hence you will need the gcc2
libraries for compatibility reasons. There are two options to get this library.
One is to compile gcc2 (with c++) in a temporary prefix (such as /var/tmp) and
then move the resulting shared libraries in /var/tmp/lib to /usr/lib. Another
option is to download the precompiled library from
<http://www.linuxfromscratch.org/~timothy/misc and move it to where the JDK can
find it.
        mv libstdc++-* $JAVA_HOME/jre/lib/i386/
Try running $JAVA_HOME/bin/java and if it reports any missing libraries, make
appropriate links. For example, if you are using gcc-2.95.3 libs, then make the
following symlink.
        ln -sf libstdc++-libc6.3-2.so.3 libstdc++-libc6.1-1.so.2


Compiling Java 2 SDK:

Unzip all the files where you plan to compile JDK:
        cd $SRC

The source files are split into two (a & b) so just combine them together using
cat:
        cat j2sdk-1_4_1-src-scsl-{a,b}.zip > j2sdk-1_4_1-src-scsl.zip

Now unzip the two files
        unzip j2sdk-1_4_1-src-scsl.zip
        unzip j2sdk-1_4_1-mozilla_headers-unix.zip

Apply the patches to fix the syntax and various other issues.
Sun includes the Motif version it needs and compiles it. To avoid linking with
the system installed libXm.a we change the name of the internal motif library
that is built. Linking against system installed libXm leads to crashes.

Now we will set some variables required by the build.
        export ALT_BOOTDIR="$JAVA_HOME"

The following variable should point to where mozilla headers are unzipped.
        export ALT_MOZILLA_PATH=$SRC

The Makefiles set the following variable to a ridiculous value. So change it.
        export ALT_DEVTOOLS_PATH="/usr/bin"

The following two variables are not required to be set, but if set they will
give the value you want when running "javac -version". I like to set them to
non-default values so that I can identify when I am switching between JDKs. Note
that these variables cannot contain spaces.
        export MILESTONE="ttcompiled" &&
        export BUILD_NUMBER=`date +%s`

We will need a root certificates file since it is not included in the source. We
use the cacerts file that is available with the precompiled j2sdk we installed
        export ALT_CACERTS_FILE=${ALT_BOOTDIR}/jre/lib/security/cacerts

Set this variable if you don't need to build the documentation and other related
stuff. People have reported problems when compiling the documentation. It
requires rpm and even then the documentation is not built. So be a good boy/girl
and do an
        export DEV_ONLY=true

There is also a variable J2RE_ONLY which if set to true builds only the Runtime
Env. Avoid setting that. Or if you like adventure, set it and try... then do a
make clobber and come back to this point.

The compilation will go awry if the following variables are set, so unset them.
        unset JAVA_HOME
        unset CLASSPATH

The build process uses two different variables to read the optimization flags.
During the first phase (hotspot build) it uses C{,XX}FLAGS while during the
second phase (j2se build) it uses OTHER_C{,XX}FLAGS. Setting CFLAGS gave me core
dumps during the second phase of the build. I unset C{,XX}FLAGS and set
OTHER_C{,XX}FLAGS to "-march=i686 -fomit-frame-pointer -s". Also unset the
LDFLAGS variable (same reason). If you set any of these variables, the build
WILL fail!!!

Using any optimizations causes miscompilation when using gcc3, especially in
java.lang.Math class. Thanks to a Gentoo user Bernhard Stegmaier
<stegmaie@db.fmi.uni-passau.de> for reporting the bug and the fix. Hence avoid
the use of any "-O?" flags. Also, some of the Makefiles hardcode "-O3", to
remove this do the following:
        for i in hotspot/build/linux/makefiles/gcc.make \
                hotspot/build/solaris/makefiles/gcc.make \
                j2se/make/sun/image/generic/Makefile
        do
                chmod +w $i
                cp $i $i.orig
                sed -e "s:\-O3:\-march=i686 \-fomit\-frame\-pointer \-s:g" \
                        $i.orig > $i
        done
For test case please refer to:
http://archive.linuxfromscratch.org/mail-archives/blfs-support/2002/11/0247.html

If you would like make to output verbose messages instead of the boring
"Compiling xxx..." messages, set the following variable:
        export MAKE_VERBOSE=true

Since you are building JDK for an unsupported platform, according to Sun you are
insane:) Hence the following variable has to be set to affirm that you really
are insane:))
        export INSANE=true

The build also looks for a Redhat Release 6.1. Though the error is not show
stopping, if you want you can make a file for the duration of the build. This is
entirely optional; and just included here for the record in case someone sends
me a bug report (if the file does not exist, Make prints a message indicating a
fatal error occured (though the compilation does not stop). Again, for the
record, I don't create this file.
        echo "Red Hat Linux release 6.1 (LFS)" > /etc/redhat-release

The source includes zlib-1.1.3 which has known security problems, so it is
prudent to upgrade the included zlib. Sun renamed some of the files in the zlib
distribution. Hence we also need to do the same. Also, linking to the dynamic
version is not trivial (I don't even know if it is possible) since the JDK build
uses some files from the zlib distribution, it doesn't build any static or
dynamic version of the zlib.
        ZLIB_VERSION=1.1.4
        cd $SRC/j2se/src/share/native/java/util/zip
        rm -rf $SRC/zlib-1.1.3
        tar -xvzf $SRC/zlib-${ZLIB_VERSION}.tar.*
        cd zlib-${ZLIB_VERSION}
        mv adler32.c zadler32.c
        mv crc32.c zcrc32.c
        cd $SRC/j2se/make/java/zip/
        cp Makefile Makefile.orig
        chmod +w Makefile
        sed -e "s:1.1.3:${ZLIB_VERSION}:" Makefile.orig > Makefile

Now the most simple part of the build.
        cd $SRC/control/make &&
        make

The build took around 4 hours on my PIII-800.

If anytime during the build, you would like to start from scratch, you can do a
"make clobber" in $SRC/control/make to go back to the beginning.

The built SDK and JRE are in the following directory.
        cd $SRC/control/build/linux-i?86

The j2sdk-image is the built JDK, congrats:)
        mkdir -p /opt/java &&
        cp -a j2sdk-image /opt/java/j2sdk-1.4.1-tt &&
        cd /opt/java &&
        ln -nsf j2sdk-1.4.0-tt j2sdk

Add the following to your </etc/profile> or <~/.profile>. Many programs expect
the JAVA_HOME variable to find the JDK.
        export JAVA_HOME=/opt/java/j2sdk
        export PATH=$PATH:$JAVA_HOME/bin

Tip: You can install multiple jdk's in /opt/java, changing between them is just
as easy as:
        ln -nsf j2sdk-<required-version> j2sdk

Link the java plugin we built for mozilla. After all that was the main reason
for me compiling jdk from scratch! Check out the mozilla hint on how to do this.
Also since we used the dynamic version of the lesstif libraries, moz will need
to preload the lesstif library, else the plugin will crash.

Now LFS CVS is all ready for big time. The first system out there to be
completely gcc3:))

If you deviate from the hint and you run into problems, first try following the
hint exactly to see if it works. Also, if you deviate from the hint and run into
successes, please let me know so that I can add the same to the hint.

Reported Problems & Workarounds:

* Harald Meier reported that if there are problems, try removing the "-z defs"
  from LDFLAGS_COMMON from line 263 in the file
  $SRC/j2se/make/common/Defs-linux.gmk.
* Parkanyi Balazs and Kelledin reported that trying to manually run java_vm
  results in unresolved libraries. Adding the relevant directories to
  /etc/ld.so.conf does not fix the problem (unless of course you have a 386
  CPU.) The solution suggested was to rename the i386 directories to i<x>86
  where <x> should be replaced by whatever your CPU is. So for a Pentium, you
  would rename the i386 directories to i586. For more info, refer to
  <http://www.cs.helsinki.fi/linux/linux-kernel/2001-25/0014.html>. Note that
  Mozilla wrapper script takes care of all this when it loads the java_vm, so
  you only have to do this if you need to run java_vm on the command line.
* Sam Halliday reported that if the partition where you are compiling runs out
  space, the build gives some very strange errors rather than the usual "Out of
  disk space" error.

CLASSPATH script:

This part is not related to compiling Java, but seemed like a nice place to put
this mini-hint:)

It is a big pain to update the CLASSPATH everytime a new package is installed. A
nice trick I use is to create a directory called /opt/classes and whenever I
install a java related package that installs a jar file, I make a link to the
relevant jar files from /opt/classes. I use the following script to
automatically create the CLASSPATH entries.
        #!/bin/sh
        # SetClassPath
        if [ -z $CLASSPATH ]
        then
                CLASSPATH=.
        else
                CLASSPATH=$CLASSPATH:.
        fi
        for i in $(ls /opt/classes/*.jar)
        do
                CLASSPATH=$CLASSPATH:$i
        done
        echo $CLASSPATH
and in /etc/profile I add the following line.
        export CLASSPATH=`SetClassPath`

Don't forget to send me bug reports and enhancements so that I can keep the hint
updated.