* more re-work

[mascara-docs.git] / i386 / junos / ucla / labs / tools.txt

[Kernel,        CS 235 Advanced Operating Systems, Fall 2010
courtesy
IowaFarmer.com  Tools
CornCam]
                You'll use two sets of tools in this class: an x86 emulator, QEMU, for running
                your kernel; and a compiler toolchain, including assembler, linker, C compiler,
                and debugger, for compiling and testing your kernel. Here's the information
                you'll need to download and install your own copies. This class assumes
                familiarity with Unix commands throughout.

                Compiler Toolchain

                A "compiler toolchain" is the set of programs, including C and C++ compilers,
                assemblers, and linkers, that turn code into executable binaries. You'll need a
                compiler toolchain that generates code for 32-bit Intel architectures ("x86"
                architectures) in the ELF binary format. We recommend you configure one of your
                machines (such as a laptop) with a working toolchain, but if you cannot do
                this, the School of Engineering's SEASnet and a private class compute server
                will be made available shortly.

                Test Your Compiler Toolchain

                Modern Linux and BSD UNIX distributions already provide a toolchain suitable
                for CS 235. To test your distribution, try the following command:

                % objdump -i

                If the second line is elf32-i386, you're all set, and don't need to compile
                your own toolchain.

                Using a Virtual Machine

                Otherwise, the easiest way to get a compatible toolchain is to install a modern
                Linux distribution on your computer. With platform virtualization, Linux can
                cohabitate with your normal computing environment. Installing a Linux virtual
                machine is a two step process. First, you download the virtualization platform.

                  • VirtualBox (free for Mac, Linux, Windows) — Download page
                  • VMware Player (free for Linux and Windows, registration required)
                  • VMware Fusion ($80 for Mac, $40 with academic discount)

                VirtualBox is a little slower and less flexible, but free!

                Once the virtualization platform is installed, download a boot disk image for
                the Linux distribution of your choice.

                  • Ubuntu Desktop is what we use.

                This will download a file named something like ubuntu-10.04.1-desktop-i386.iso.
                Start up your virtualization platform and create a new (32-bit) virtual
                machine. Use the downloaded Ubuntu image as a boot disk; the procedure differs
                among VMs but is pretty simple. Type objdump -i, as above, to verify that your
                toolchain is now set up. You will do your work inside the VM.

                Building Your Own Toolchain

                This will take longer to set up, but give slightly better performance than a
                virtual machine, and lets you work in your own familiar environment. (Mac/Unix
                Only)

                We assume that you are installing the toolchain into /usr/local. You will need
                a fair amount of disk space to compile the tools (around 1GiB). If you don't
                have that much space, delete each directory after its make install step.

                Download the following packages:

                  • ftp://ftp.gmplib.org/pub/gmp-5.0.1/gmp-5.0.1.tar.bz2
                  • http://www.mpfr.org/mpfr-current/mpfr-3.0.0.tar.bz2
                  • http://www.multiprecision.org/mpc/download/mpc-0.8.2.tar.gz
                  • http://ftpmirror.gnu.org/binutils/binutils-2.20.1.tar.bz2
                  • http://ftpmirror.gnu.org/gcc/gcc-4.5.1/gcc-core-4.5.1.tar.bz2
                  • http://ftpmirror.gnu.org/gcc/gcc-4.5.1/gcc-g++-4.5.1.tar.bz2
                  • http://ftpmirror.gnu.org/gdb/gdb-7.2.tar.bz2

                Unpack and build the packages. The green bold text shows you how to install
                into /usr/local, which is what we recommend. To install into a different
                directory, $PFX, click herenote the differences in lighter type (hide). If you
                have problems, see below.

                export PATH=$PFX/bin:$PATH
                export LD_LIBRARY_PATH=$PFX/lib:$LD_LIBRARY_PATH      # On Mac OS X, use DYLD_LIBRARY_PATH

                tar xjf gmp-5.0.1.tar.bz2
                cd gmp-5.0.1
                ./configure --prefix=/usr/local--prefix=$PFX
                make
                make install             # This step may require privilege (sudo make install)
                cd ..

                tar xjf mpfr-3.0.0.tar.bz2
                cd mpfr-3.0.0
                ./configure --prefix=/usr/local--prefix=$PFX --with-gmp=$PFX
                make
                make install             # This step may require privilege (sudo make install)
                cd ..

                tar xzf mpc-0.8.2.tar.gz
                cd mpc-0.8.2
                ./configure --prefix=/usr/local--prefix=$PFX --with-gmp=$PFX --with-mpfr=$PFX
                make
                make install             # This step may require privilege (sudo make install)
                cd ..


                tar xjf binutils-2.20.1.tar.bz2
                cd binutils-2.20.1
                ./configure --prefix=/usr/local--prefix=$PFX --target=i386-jos-elf --disable-werror
                make
                make install             # This step may require privilege (sudo make install)
                cd ..

                i386-jos-elf-objdump -i
                # Should produce output like:
                # BFD header file version (GNU Binutils) 2.20.1.20100303
                # elf32-i386
                #  (header little endian, data little endian)
                #   i386...


                tar xjf gcc-core-4.5.1.tar.bz2
                tar xjf gcc-g++-4.5.1.tar.bz2
                cd gcc-4.5.1
                mkdir build              # GCC will not compile correctly unless you build in a separate directory
                cd build
                ../configure --prefix=/usr/local--prefix=$PFX --with-gmp=$PFX --with-mpfr=$PFX --with-mpc=$PFX \
                    --target=i386-jos-elf --disable-werror \
                    --disable-libssp --disable-libmudflap --with-newlib \
                    --without-headers --enable-languages=c,c++
                make all-gcc
                make install-gcc         # This step may require privilege (sudo make install-gcc)
                make all-target-libgcc
                make install-target-libgcc     # This step may require privilege (sudo make install-target-libgcc)
                cd ../..

                i386-jos-elf-gcc -v
                # Should produce output like:
                # Using built-in specs.
                # COLLECT_GCC=i386-jos-elf-gcc
                # COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/i386-jos-elf/4.5.1/lto-wrapper
                # Target: i386-jos-elf


                tar xjf gdb-7.2.tar.bz2
                cd gdb-7.2
                ./configure --prefix=/usr/local--prefix=$PFX --target=i386-jos-elf --program-prefix=i386-jos-elf- \
                    --disable-werror
                make all
                make install             # This step may require privilege (sudo make install)
                cd ..

                Troubleshooting

                Q. I can't run make install because I don't have root permission on this
                    machine.
                    A. Our instructions assume you are installing into the /usr/local
                    directory. However, this may not be allowed in your environment. If you can
                    only install code into your home directory, that's OK. In the instructions
                    above, replace --prefix=/usr/local with --prefix=$HOME (and click here to
                    update the instructions further). You will also need to change your PATH
                    and LD_LIBRARY_PATH (Linux) or DYLD_LIBRARY_PATH (Mac OS X) environment
                    variables, to inform your shell where to find the tools. For example:

                    export PATH=$HOME/bin:$PATH
                    export LD_LIBRARY_PATH=$HOME/lib:$LD_LIBRARY_PATH

                    Enter these lines in your ~/.bashrc file so you don't need to type them
                    every time you log in.

                Q. My build fails with an inscrutable message about "library not found".
                    A. You need to set your LD_LIBRARY_PATH (Linux) or DYLD_LIBRARY_PATH (Mac
                    OS X) environment variable to include the libraries you built; see above.
                    The environment variable must include the PREFIX/lib directory (for
                    instance, /usr/local/lib).

                QEMU Emulator

                We provide a patched version of the QEMU emulator with improved support for
                debugging. (Credit to Kohler and Austin Clements and others at MIT 6.828.) We
                recommend you compile QEMU from this source; if you use the version that came
                installed on your Linux distribution, you will not be able to complete all the
                debugging exercises.

                First, download the source:

                  • http://www.cs.ucla.edu/~kohler/class/10f-aos/qemu-0.12.5-aos.tar.bz2

                On a Linux machine, you will need to install the SDL graphics library to build
                QEMU. On Ubuntu or Debian:

                sudo aptitude install libsdl1.2-dev

                Then unpack, compile, and install QEMU as follows.

                tar xjf qemu-0.12.5-aos.tar.bz2
                cd qemu-0.12.5-aos
                On Mac OS X:
                   ./configure --prefix=/usr/local --disable-sdl --enable-cocoa \
                      --disable-docs --target-list="i386-softmmu x86_64-softmmu"
                On Linux:
                   ./configure --prefix=/usr/local --target-list="i386-softmmu x86_64-softmmu"
                make
                make install             # This step may require privilege (sudo make install)

                The QEMU monitor is accessible by pressing Ctrl-Alt-2 inside the QEMU window.
                Return to normal display with Ctrl-Alt-1. QEMU tends to take control of your
                mouse. If you can't find a mouse pointer, check the QEMU title bar for text
                like "Press Ctrl-Alt to exit grab" and, if so, press Ctrl-Alt to regain
                control.

                Source Code Control

                You will do your work using a source code control system. This will help you
                maintain two parallel branches—one with your changes, and one with the labs as
                we release them. We will support two source code control systems: CVS and Git.
                Git is by far the better system, and strongly preferred by your instructor for
                his own work, but this is the first time we're trying it in CS 235, so it will
                be more experimental.

                You don't need to compile CVS or Git yourself. The packaged versions that come
                with your OS will be good enough. You just need to install them.

                  • Ubuntu/Debian Linux: Run sudo aptitude install cvs git-core
                  • Mac OS X: Install MacPorts, then run sudo port install cvs git-core
                  • Windows: Versions are available; contact the instructor if you need help

                Using Cygwin on a Windows Machine

                If you have a Windows machine, but cannot run a Linux virtual machine or
                install a dual-boot configuration, the easiest way to work on labs from home
                will be to use a class compute server or SEASnet coupled with VNC remote
                display. Let us know as soon as possible if you want this configuration to
                work.

                Nevertheless, it should be possible to execute many of the above steps and get
                a functional build environment running on Windows using Cygwin. Here's how to
                install Cygwin on your Windows machine.

                 1. Install Cygwin/X. Follow this procedure. Make sure you select the
                    "xorg-x11-base" package and the "openssh" package. Also select the "flex"
                    and "bison" packages from the development header. This sets up a Unix-like
                    environment on your Windows machine, including an X server.
                 2. Start the Cygwin X server by running startxwin.bat. "Run /usr/X11R6/bin/
                    startxwin.bat by double-clicking it in Windows Explorer." http://
                    x.cygwin.com/docs/ug/using.html

                You should also be able to compile the class tools and the labs on your Windows
                machine, once Cygwin has been installed. Let us know if you have success with
                this.

                Using the Class Compute Server

                A compute server is available for your use. Request a login from your
                instructor; your mail should include your preferred username and an SSH public
                key (use PuTTY on Windows [PuTTY authentication notes]).

                Using VNC

                VNC is a compressed remote desktop viewing protocol that QEMU understands.
                Using VNC will be faster; QEMU is very slow over a remote X connection. You
                will need a VNC viewer on your local computer. Good, free viewers are available
                for Linux, Mac OS X, and Windows.

                When the instructor creates your account, you'll receive a VNC port allocation
                (something like 5500). Remember this number; we call it VNCPORT below.

                  • Download a VNC viewer. On Linux, we have tested TightVNC (Ubuntu/Debian
                    package xtightvncviewer). On Mac OS X, try Chicken of the VNC (an installer
                    disk image is available). On Windows, TightVNC also works.
                  • Start your VNC viewer in “listening” mode, on port 5500 (display 0). On
                    Linux with TightVNC, try

                    vncviewer -listen 0 -encodings "copyrect zlib raw"

                    (You can leave off the -encodings argument, but display will be slower.) On
                    Mac OS X with Chicken, try "Connection > Listen for Server...", and enter
                    5500.
                  • Log on to the class server, forwarding the server's VNCPORT to your local
                    port 5500. This will let the server QEMU display into your local VNC
                    window. Sample command:

                    ssh -RVNCPORT:localhost:5500 seek.cs.ucla.edu

                  • When running QEMU on the server using make, supply the argument VNC=VNCPORT
                    . For example:

                    make run VNC=VNCPORT

                Using X windows

                This may be simpler if you have trouble finding a VNC viewer. Simply start up
                an X server and run

                ssh -Y seek.cs.ucla.edu

                Do your work as usual. The -Y option will forward QEMU's window onto your local
                display. Try make run-nox if you don't need the QEMU window.

                Back to CS 235 Advanced Operating Systems