NetBSD Advent Calendar 2017の5日目の記事です。
はじめに
新しいMozilla Firefoxのビルドには、Rustプログラミング言語のコンパイラーであるrustcと、パッケージ管理プログラムのcargoが必要となっている。
jakllschさんがNetBSD/amd64には移植してくれているので、RustのアップストリームがNetBSD/amd64 7向けのブートストラップキットはビルドしてくれている。
だが、NetBSD/i386 7向けがないと、Mozilla FirefoxをNetBSD/i386でビルドできない。
私はhttps://deuterium.ryoon.net/pub/rust/でnetBSD/i386 7用のブートストラップキット
を公開しているが、信頼できるか分からない人の作ったコンパイラーなど使うべきではない。
ここにブートストラップキットの作り方を書いておくので、自分のためのブートストラップキットを作って使って欲しい。
# RustのアップストリームにどうやってNetBSD/i386用をビルドしてもらうように交渉すれば良いのか…。
NetBSD/amd64の環境を用意する
NetBSD/amd64 7.0以降の環境を用意する。Rustのアップストリームのビルドしているブートストラップキットが動けばNetBSD/amd64のバージョンは問わない。
ここでは、NetBSD/amd64 7.1の環境があったので、それを使用することにした。
ところで、原理的にはNetBSD/amd64である必要もない。
ある程度POSIX準拠なオペレーティングシステムで、Rustがサポートしブートストラップキットを提供しているアーキテクチャーであれば良い。
次に説明するNetBSD/i386のクロスツールチェインはそこでビルドできるはずである。
NetBSD/i386 7用のクロスツールチェインとユーザーランドをビルドする
NetBSDはクロスツールチェインを簡単に生成でき、クロスコンパイルをする準備が簡単にできるのが良い所である。
まず、以下のように実行してクロスコンパイラーなどのツールチェインを作成する。
これは、NetBSD/i386のユーザーランドをビルドするのに使用するだけでなく、Rustブートストラップキットをビルドするのにも使用する。
ここで、/usr/srcにnetbsd-7のツリーを用意してあるとする。
$ cd /usr/src
$ ./build.sh -m i386 -U -T /usr/world/7/i386/tools -O /usr/world/7/i386/obj -D /usr/world/7/i386/destdir tools
$ ./build.sh -m i386 -U -T /usr/world/7/i386/tools -O /usr/world/7/i386/obj -D /usr/world/7/i386/destdir build
これで、
/usr/world/7/i386/tools以下にクロスツールチェインが用意され、
それを使って
/usr/world/7/i386/destdir以下にユーザーランドがビルドされインストールされた。
クロスツールチェインを改造する
Rustブートストラップキットに使用するクロスコンパイラーのgccとg++には、いくつかのコマンドライン引数を与えて動かす必要があるのだが、
Rustのブートストラップメカニズムはそのようなコマンドライン引数を与えてgcc/g++を呼び出してはくれない。
なので、必要なコマンドライン引数をあらかじめ指定したシェルスクリプトを用意し、それをクロスコンパイラーの代わりに使用することにする。
以下のようにi486--netbsdelf-g++-sysrootとi486--netbsdelf-gcc-sysrootを用意しておく。
$ cat /usr/world/7/i386/tools/bin/i486--netbsdelf-g++-sysroot
#! /bin/sh
/usr/world/7/i386/tools/bin/i486--netbsdelf-g++ --sysroot=/usr/world/7/i386/destdir -march=i586 "$@"
$ cat /usr/world/7/i386/tools/bin/i486--netbsdelf-gcc-sysroot
#! /bin/sh
/usr/world/7/i386/tools/bin/i486--netbsdelf-gcc --sysroot=/usr/world/7/i386/destdir "$@"
ここで、
-march=i586は、atomic ops関連の関数がundefined referenceにならないために必要だったはずである。
NetBSD/i386のユーザーランドに必要なパッケージをインストールする
/usr/world/7/i386/destdirにユーザーランドが用意されているが、Rustブートストラップキットをビルドするには、
pkgsrc/lang/rustが必要としているような
ライブラリーが必要になる。
llvm関係はRustのソースコードに含まれているので、それ以外を/usr/world/7/i386/destdirにインストールしておく。
NetBSD/amd64では、NetBSD/i386のバイナリーを実行できるので、ここではchroot(8)コマンドを使って、ライブラリーを準備してしまった。
これは安直な方法なので、例えば今後NetBSD/earm向けのRustブートストラップキットを作ろうと言う場合には、この方法は採用できない。
必要なライブラリーをクロスビルドして配置するか、NetBSD/earmでビルドとインストールしたユーザーランドをコピーして来る必要があるだろう。
# chroot /usr/world/7/i386/destdir /bin/ksh
# pkg_add libgit2-0.26.0nb1.tgz
; 手作業でlibhttp_parser.aをビルドし、/usr/libに配置する。
; 合わせて、http_parser.hを/usr/includeに配置する。
# exit
ここで、
http-parserは、
/usr/pkg以下にあってはビルドに失敗した。
面倒だったので
/usr以下に配置したが、本来であればRustで書かれたRustのビルドメカニズムを修正すべきであろう。
NetBSD/amd64 7用のRustをビルドできるようにする
この先、NetBSD/amd64用のrustcやcargoをビルドし、それを使ってNetBSD/i386 7用のrustcやcargoをビルドしていくのである。
当然だが、NetBSD/amd64向けにpkgsrc/lang/rust
が必要とするライブラリー等が用意されている必要がある。
以下のように実行すれば、/usr/pkg以下にインストールされる。
ついでに、Rustブートストラップキットが必要とするGNU coreutils
とOpenSSL用のルート証明書、
ninjaもインストールしてしまおう。
ninjaは、lllvmの部分をパラレルビルドするのに必要となる。
# cd /usr/pkgsrc/lang/rust
# make depends
# cd /usr/pkgsrc/sysutils/coreutils
# make install
# cd /usr/pkgsrc/security/mozilla-rootcerts-openssl
# make install
Rustブートストラップキットを展開する
基本的には、一つ前のリリースのブートストラップキットを使って、最新のリリースのブートストラップキットを作るのだと思う。
ここでは、Rust 1.22.1をNetBSD/i386 7でビルドしようとしているので、1.21.0のブートストラップキットを作ることになる。
その1.21.0のブートストラップキットをビルドするために使用するRustアップストリームのブートストラップキットは、1.20.0となる。
以下のように展開しておく。
$ mkdir ~/tmp
$ cd ~/tmp
$ ftp http://static.rust-lang.org/dist/rust-1.21.0-x86_64-unknown-netbsd.tar.gz
$ ftp http://static.rust-lang.org/dist/rust-std-1.20.0-i686-unknown-netbsd.tar.gz
$ tar zxvf rust-1.21.0-x86_64-unknown-netbsd.tar.gz
$ tar zxvf rust-std-1.20.0-i686-unknown-netbsd.tar.gz
$ mv rust-std-1.20.0-x86_64-unknown-netbsd/rust-std-x86_64-unknown-netbsd/lib/rustlib/x86_64-unknown-netbsd \
rust-1.20.0-x86_64-unknown-netbsd/rustc/lib/rustlib/
$ rm -r rust-std-1.20.0-x86_64-unknown-netbsd
NetBSD/i386のためのパッチを適用する
NetBSD/i386のRustブートストラップキットをビルドするのに必要なパッチを適用する。
そのパッチは以下のようである。
--- src/bootstrap/native.rs
+++ src/bootstrap/native.rs
@@ -407,6 +407,7 @@ impl Step for Openssl {
"i686-unknown-freebsd" => "BSD-x86-elf",
"i686-unknown-linux-gnu" => "linux-elf",
"i686-unknown-linux-musl" => "linux-elf",
+ "i686-unknown-netbsd" => "BSD-x86-elf",
"mips-unknown-linux-gnu" => "linux-mips32",
"mips64-unknown-linux-gnuabi64" => "linux64-mips64",
"mips64el-unknown-linux-gnuabi64" => "linux64-mips64",
NetBSD/amd64 7のためのパッチを適用する
NetBSD/amd64 7で動くllvmのバイナリーを得るには、以下のパッチが必要であった。NetBSD/amd64 8以降では不要らしい。
--- src/llvm/cmake/modules/AddLLVM.cmake
+++ src/llvm/cmake/modules/AddLLVM.cmake
@@ -1422,7 +1422,7 @@ function(llvm_setup_rpath name)
set(_install_name_dir INSTALL_NAME_DIR "@rpath")
set(_install_rpath "@loader_path/../lib" ${extra_libdir})
elseif(UNIX)
- set(_install_rpath "\$ORIGIN/../lib${LLVM_LIBDIR_SUFFIX}" ${extra_libdir})
+# set(_install_rpath "\$ORIGIN/../lib${LLVM_LIBDIR_SUFFIX}" ${extra_libdir})
if(${CMAKE_SYSTEM_NAME} MATCHES "(FreeBSD|DragonFly)")
set_property(TARGET ${name} APPEND_STRING PROPERTY
LINK_FLAGS " -Wl,-z,origin ")
config.tomlにビルド条件を設定する
以下のようにi686-unknown-netbsdブートストラップキットを作ること、その他のコマンドのパス等々を設定する。
# Sample TOML configuration file for building Rust.
#
# To configure rustbuild, copy this file to the directory from which you will be
# running the build, and name it config.toml.
#
# All options are commented out by default in this file, and they're commented
# out with their default values. The build system by default looks for
# `config.toml` in the current directory of a build for build configuration, but
# a custom configuration file can also be specified with `--config` to the build
# system.
# =============================================================================
# Tweaking how LLVM is compiled
# =============================================================================
[llvm]
# Indicates whether rustc will support compilation with LLVM
# note: rustc does not compile without LLVM at the moment
#enabled = true
# Indicates whether the LLVM build is a Release or Debug build
#optimize = true
# Indicates whether an LLVM Release build should include debug info
#release-debuginfo = false
# Indicates whether the LLVM assertions are enabled or not
#assertions = false
# Indicates whether ccache is used when building LLVM
#ccache = false
# or alternatively ...
#ccache = "/path/to/ccache"
# If an external LLVM root is specified, we automatically check the version by
# default to make sure it's within the range that we're expecting, but setting
# this flag will indicate that this version check should not be done.
#version-check = true
# Link libstdc++ statically into the librustc_llvm instead of relying on a
# dynamic version to be available.
static-libstdcpp = true
# Tell the LLVM build system to use Ninja instead of the platform default for
# the generated build system. This can sometimes be faster than make, for
# example.
ninja = true
# LLVM targets to build support for.
# Note: this is NOT related to Rust compilation targets. However, as Rust is
# dependent on LLVM for code generation, turning targets off here WILL lead to
# the resulting rustc being unable to compile for the disabled architectures.
# Also worth pointing out is that, in case support for new targets are added to
# LLVM, enabling them here doesn't mean Rust is automatically gaining said
# support. You'll need to write a target specification at least, and most
# likely, teach rustc about the C ABI of the target. Get in touch with the
# Rust team and file an issue if you need assistance in porting!
#targets = "X86;ARM;AArch64;Mips;PowerPC;SystemZ;JSBackend;MSP430;Sparc;NVPTX;Hexagon"
# LLVM experimental targets to build support for. These targets are specified in
# the same format as above, but since these targets are experimental, they are
# not built by default and the experimental Rust compilation targets that depend
# on them will not work unless the user opts in to building them. Possible
# experimental LLVM targets include WebAssembly for the
# wasm32-experimental-emscripten Rust target.
#experimental-targets = ""
# Cap the number of parallel linker invocations when compiling LLVM.
# This can be useful when building LLVM with debug info, which significantly
# increases the size of binaries and consequently the memory required by
# each linker process.
# If absent or 0, linker invocations are treated like any other job and
# controlled by rustbuild's -j parameter.
#link-jobs = 0
# When invoking `llvm-config` this configures whether the `--shared` argument is
# passed to prefer linking to shared libraries.
#link-shared = false
# =============================================================================
# General build configuration options
# =============================================================================
[build]
# Build triple for the original snapshot compiler. This must be a compiler that
# nightlies are already produced for. The current platform must be able to run
# binaries of this build triple and the nightly will be used to bootstrap the
# first compiler.
build = "x86_64-unknown-netbsd" # defaults to your host platform
# In addition to the build triple, other triples to produce full compiler
# toolchains for. Each of these triples will be bootstrapped from the build
# triple and then will continue to bootstrap themselves. This platform must
# currently be able to run all of the triples provided here.
host = ["i686-unknown-netbsd"] # defaults to just the build triple
# In addition to all host triples, other triples to produce the standard library
# for. Each host triple will be used to produce a copy of the standard library
# for each target triple.
target = ["i686-unknown-netbsd"] # defaults to just the build triple
# Instead of downloading the src/stage0.txt version of Cargo specified, use
# this Cargo binary instead to build all Rust code
cargo = "/home/ryoon/tmp/rust-1.20.0-x86_64-unknown-netbsd/cargo/bin/cargo"
# Instead of downloading the src/stage0.txt version of the compiler
# specified, use this rustc binary instead as the stage0 snapshot compiler.
rustc = "/home/ryoon/tmp/rust-1.20.0-x86_64-unknown-netbsd/rustc/bin/rustc"
# Flag to specify whether any documentation is built. If false, rustdoc and
# friends will still be compiled but they will not be used to generate any
# documentation.
docs = true
# Indicate whether the compiler should be documented in addition to the standard
# library and facade crates.
compiler-docs = true
# Indicate whether submodules are managed and updated automatically.
#submodules = true
# The path to (or name of) the GDB executable to use. This is only used for
# executing the debuginfo test suite.
#gdb = "gdb"
# The node.js executable to use. Note that this is only used for the emscripten
# target when running tests, otherwise this can be omitted.
#nodejs = "node"
# Python interpreter to use for various tasks throughout the build, notably
# rustdoc tests, the lldb python interpreter, and some dist bits and pieces.
# Note that Python 2 is currently required.
python = "/usr/pkg/bin/python2.7"
# Force Cargo to check that Cargo.lock describes the precise dependency
# set that all the Cargo.toml files create, instead of updating it.
#locked-deps = false
# Indicate whether the vendored sources are used for Rust dependencies or not
vendor = true
# Typically the build system will build the rust compiler twice. The second
# compiler, however, will simply use its own libraries to link against. If you
# would rather to perform a full bootstrap, compiling the compiler three times,
# then you can set this option to true. You shouldn't ever need to set this
# option to true.
full-bootstrap = false
# Enable a build of the extended rust tool set which is not only the compiler
# but also tools such as Cargo. This will also produce "combined installers"
# which are used to install Rust and Cargo together. This is disabled by
# default.
extended = true
# Verbosity level: 0 == not verbose, 1 == verbose, 2 == very verbose
#verbose = 0
# Build the sanitizer runtimes
#sanitizers = false
# Build the profiler runtime
#profiler = false
# Indicates whether the OpenSSL linked into Cargo will be statically linked or
# not. If static linkage is specified then the build system will download a
# known-good version of OpenSSL, compile it, and link it to Cargo.
openssl-static = true
# Run the build with low priority, by setting the process group's "nice" value
# to +10 on Unix platforms, and by using a "low priority" job object on Windows.
#low-priority = false
# Arguments passed to the `./configure` script, used during distcheck. You
# probably won't fill this in but rather it's filled in by the `./configure`
# script.
#configure-args = []
# Indicates that a local rebuild is occurring instead of a full bootstrap,
# essentially skipping stage0 as the local compiler is recompiling itself again.
#local-rebuild = false
# =============================================================================
# General install configuration options
# =============================================================================
[install]
# Instead of installing to /usr/local, install to this path instead.
#prefix = "/usr/local"
# Where to install system configuration files
# If this is a relative path, it will get installed in `prefix` above
#sysconfdir = "/etc"
# Where to install documentation in `prefix` above
#docdir = "share/doc/rust"
# Where to install binaries in `prefix` above
#bindir = "bin"
# Where to install libraries in `prefix` above
#libdir = "lib"
# Where to install man pages in `prefix` above
#mandir = "share/man"
# Where to install data in `prefix` above (currently unused)
#datadir = "share"
# Where to install additional info in `prefix` above (currently unused)
#infodir = "share/info"
# Where to install local state (currently unused)
# If this is a relative path, it will get installed in `prefix` above
#localstatedir = "/var/lib"
# =============================================================================
# Options for compiling Rust code itself
# =============================================================================
[rust]
# Indicates that the build should be optimized for debugging Rust. Note that
# this is typically not what you want as it takes an incredibly large amount of
# time to have a debug-mode rustc compile any code (notably libstd). If this
# value is set to `true` it will affect a number of configuration options below
# as well, if unconfigured.
#debug = false
# Whether or not to optimize the compiler and standard library
# Note: the slowness of the non optimized compiler compiling itself usually
# outweighs the time gains in not doing optimizations, therefore a
# full bootstrap takes much more time with optimize set to false.
#optimize = true
# Number of codegen units to use for each compiler invocation. A value of 0
# means "the number of cores on this machine", and 1+ is passed through to the
# compiler.
#codegen-units = 1
# Whether or not debug assertions are enabled for the compiler and standard
# library. Also enables compilation of debug! and trace! logging macros.
#debug-assertions = false
# Whether or not debuginfo is emitted
#debuginfo = false
# Whether or not line number debug information is emitted
#debuginfo-lines = false
# Whether or not to only build debuginfo for the standard library if enabled.
# If enabled, this will not compile the compiler with debuginfo, just the
# standard library.
#debuginfo-only-std = false
# Whether or not jemalloc is built and enabled
#use-jemalloc = true
# Whether or not jemalloc is built with its debug option set
#debug-jemalloc = false
# Whether or not `panic!`s generate backtraces (RUST_BACKTRACE)
#backtrace = true
# The default linker that will be hard-coded into the generated compiler for
# targets that don't specify linker explicitly in their target specifications.
# Note that this is not the linker used to link said compiler.
#default-linker = "cc"
# The "channel" for the Rust build to produce. The stable/beta channels only
# allow using stable features, whereas the nightly and dev channels allow using
# nightly features
channel = "stable"
# By default the `rustc` executable is built with `-Wl,-rpath` flags on Unix
# platforms to ensure that the compiler is usable by default from the build
# directory (as it links to a number of dynamic libraries). This may not be
# desired in distributions, for example.
#rpath = true
# Suppresses extraneous output from tests to ensure the output of the test
# harness is relatively clean.
#quiet-tests = false
# Flag indicating whether tests are compiled with optimizations (the -O flag) or
# with debuginfo (the -g flag)
#optimize-tests = true
#debuginfo-tests = true
# Flag indicating whether codegen tests will be run or not. If you get an error
# saying that the FileCheck executable is missing, you may want to disable this.
#codegen-tests = true
# Flag indicating whether git info will be retrieved from .git automatically.
# Having the git information can cause a lot of rebuilds during development.
# Note: If this attribute is not explicity set (e.g. if left commented out) it
# will default to true if channel = "dev", but will default to false otherwise.
#ignore-git = true
# When creating source tarballs whether or not to create a source tarball.
#dist-src = false
# Whether to also run the Miri tests suite when running tests.
# As a side-effect also generates MIR for all libraries.
#test-miri = false
# =============================================================================
# Options for specific targets
#
# Each of the following options is scoped to the specific target triple in
# question and is used for determining how to compile each target.
# =============================================================================
[target.i686-unknown-netbsd]
# C compiler to be used to compiler C code. Note that the
# default value is platform specific, and if not specified it may also depend on
# what platform is crossing to what platform.
#cc = "cc"
# C++ compiler to be used to compiler C++ code (e.g. LLVM and our LLVM shims).
# This is only used for host targets.
#cxx = "c++"
# Archiver to be used to assemble static libraries compiled from C/C++ code.
# Note: an absolute path should be used, otherwise LLVM build will break.
#ar = "ar"
# Linker to be used to link Rust code. Note that the
# default value is platform specific, and if not specified it may also depend on
# what platform is crossing to what platform.
#linker = "cc"
# Path to the `llvm-config` binary of the installation of a custom LLVM to link
# against. Note that if this is specifed we don't compile LLVM at all for this
# target.
#llvm-config = "../path/to/llvm/root/bin/llvm-config"
# Path to the custom jemalloc static library to link into the standard library
# by default. This is only used if jemalloc is still enabled above
#jemalloc = "/path/to/jemalloc/libjemalloc_pic.a"
# If this target is for Android, this option will be required to specify where
# the NDK for the target lives. This is used to find the C compiler to link and
# build native code.
#android-ndk = "/path/to/ndk"
# Force static or dynamic linkage of the standard library for this target. If
# this target is a host for rustc, this will also affect the linkage of the
# compiler itself. This is useful for building rustc on targets that normally
# only use static libraries. If unset, the target's default linkage is used.
#crt-static = false
# The root location of the MUSL installation directory. The library directory
# will also need to contain libunwind.a for an unwinding implementation. Note
# that this option only makes sense for MUSL targets that produce statically
# linked binaries
#musl-root = "..."
# Used in testing for configuring where the QEMU images are located, you
# probably don't want to use this.
#qemu-rootfs = "..."
# =============================================================================
# Distribution options
#
# These options are related to distribution, mostly for the Rust project itself.
# You probably won't need to concern yourself with any of these options
# =============================================================================
[dist]
# This is the folder of artifacts that the build system will sign. All files in
# this directory will be signed with the default gpg key using the system `gpg`
# binary. The `asc` and `sha256` files will all be output into the standard dist
# output folder (currently `build/dist`)
#
# This folder should be populated ahead of time before the build system is
# invoked.
#sign-folder = "path/to/folder/to/sign"
# This is a file which contains the password of the default gpg key. This will
# be passed to `gpg` down the road when signing all files in `sign-folder`
# above. This should be stored in plaintext.
#gpg-password-file = "path/to/gpg/password"
# The remote address that all artifacts will eventually be uploaded to. The
# build system generates manifests which will point to these urls, and for the
# manifests to be correct they'll have to have the right URLs encoded.
#
# Note that this address should not contain a trailing slash as file names will
# be appended to it.
#upload-addr = "https://example.com/folder"
# Whether to build a plain source tarball to upload
# We disable that on Windows not to override the one already uploaded on S3
# as the one built on Windows will contain backslashes in paths causing problems
# on linux
#src-tarball = true
Rustブートストラップキットをビルドする
ここまで来れば、後は実行してしばらく待つだけである。私のところのマシンでは正確には把握していないが数時間は必要だった。
以下のように実行すれば良い。
$ mkdir ~/tmp/rust
$ cd ~/tmp/rust
$ LD_LIBRARY_PATH=/home/ryoon/tmp/rust-1.20.0-x86_64-unknown-netbsd/rustc/lib OPENSSL_DIR=/usr CC_i686_unknown_netbsd=i486--netbsdelf-gcc-sysroot CXX_i686_unknown_netbsd=i486--netbsdelf-g++-sysroot PATH=/usr/world/7/i386/tools/bin:/usr/pkg/gnu/bin:$PATH python2.7 ../rustc-1.21.0-src/x.py dist
エラーなく完了したら、以下のようにブートストラップキットができているはずである。
$ cd ~/tmp/rust
$ ls build/dist
(snip)
build/dist/rust-1.19.0-i686-unknown-netbsd.tar.gz
(snip)
build/dist/rust-std-1.19.0-i686-unknown-netbsd.tar.gz
(snip)
これをDISTDIRに置いて、
pkgsrc/lang/rustをNetBSD/i386でビルドするのに使えば良い。
