|
Getting started with the Userland Consolidation
|
|
|
Getting Started
|
|
This README provides a very brief overview of the gate, how to retrieve
|
a copy, and how to build it. Detailed documentation about the Userland
|
gate can be found in the 'doc' directory. Questions or comments about
|
the gate can be addressed to oi-dev@openindiana.org.
|
|
Overview
|
|
The Userland consolidation maintains a Git repository at
|
|
https://github.com/OpenIndiana/oi-userland
|
|
This gate contains build recipies, patches, IPS manifests, etc. necessary
|
to download, prep, build, test, package and publish open source software.
|
The build infrastructure is similiar to that of the SFW consolidation in
|
that it makes use of herarchical Makefiles which provide dependency and
|
recipe information for building the components. In order to build the
|
contents of the Userland gate, you need to clone it. Since you are
|
reading this, you probably already have.
|
|
Getting the Bits
|
|
As mentioned, the gate is stored in a Git repository. In order to
|
build or develop in the gate, you will need to clone it. You can do so
|
with the following command
|
|
$ git clone https://github.com/OpenIndiana/oi-userland.git /scratch/clone
|
|
This will create a replica of the various pieces that are checked into the
|
source code management system, but it does not retrieve the community
|
source archives associated with the gate content. To download the
|
community source associated with your cloned workspace, you will need to
|
execute the following:
|
|
$ cd /scratch/clone/components
|
$ gmake download
|
|
This will use GNU make and the downloading tool in the gate to walk through
|
all of the component directories downloading and validating the community
|
source archives from the gate machine or their canonical source repository.
|
|
There are two variation to this that you may find interesting. First, you
|
can cause gmake(1) to perform it's work in parallel by adding '-j (jobs)'
|
to the command line. Second, if you are only interested in working on a
|
particular component, you can change directories to that component's
|
directory and use 'gmake download' from that to only get it's source
|
archive.
|
|
Also, when you start to work with a new archive file - update the source
|
version in an existing recipe component, or start a new one from scratch -
|
you can use 'gmake fetch' to download the archive(s) defined in the new
|
recipe, calculate the checksums and *NOT* remove the archive because its
|
actual checksum does not match the value recorded in the recipe Makefile
|
(if any) so the download is deemed corrupted while you know it is not.
|
There is also a side-effect: by framework recipe, a file in the download
|
location only depends on the component recipe Makefile. So once an archive
|
is "fetched" (downloaded and not removed), it will not be re-verified -
|
the downloading script is just not called. This is a moderate problem,
|
since the "fetch" ability is a helper for recipe-makers doing initial
|
archive downloads in a certain situation, to save some traffic and time
|
on their workstations. You can still remove files fetched by a recipe
|
using 'gmake clobber'.
|
|
Building the Bits
|
|
You can build individual components or the contents of the entire gate.
|
|
Integration with ccache to speed up re-builds
|
|
If you happen to build the same sources several times (e.g. iterating
|
attempts to produce a working recipe, or maintaining an automated build
|
server), you can benefit from 'ccache' integration in 'oi-userland'.
|
Note that this feature is currently experimental and off by default.
|
|
The 'ccache' component is available as part of the project repository.
|
Once you have the resulting package installed, you can pass the 'make'
|
argument or environment variable 'ENABLE_CCACHE=true' to wrap the GNU
|
compiler invocations with the caching program - so that the same inputs
|
would re-produce same outputs quickly.
|
|
You can pre-set this variable in your user account '~/.profile' like this:
|
|
### To speed up OI-userland re-builds
|
ENABLE_CCACHE=true
|
export ENABLE_CCACHE
|
|
Note: be wary of ccache's own CCACHE_DISABLE environment variable: any
|
value (empty, "false" etc.) is considered a "true" setting for ccache
|
booleans (and so disables the program, falling through to real compiler).
|
|
Keeping all sources in one place
|
|
The Userland consolidation tools automate download of required source
|
tarballs. By older default they were kept in each component's directory,
|
but you could centralize it by using the 'USERLAND_ARCHIVES' variable.
|
Recently the defaults change to pre-initialize 'USERLAND_ARCHIVES' to
|
point into '$(WS_TOP)/archives/' unless customized by the caller - for
|
example, to share the common download area between multiple workspaces.
|
|
You can pre-set this variable in your user account '~/.profile' like
|
this (and note that the trailing slash is required):
|
|
### For oi-userland source files
|
USERLAND_ARCHIVES="$HOME/Downloads/"
|
export USERLAND_ARCHIVES
|
|
See also the 'make-rules/shared-macros.mk' for 'INTERNAL_ARCHIVE_MIRROR',
|
'EXTERNAL_ARCHIVE_MIRROR' and envvar 'DOWNLOAD_SEARCH_PATH' to get some
|
ideas about using HTTP mirrors to e.g. reduce network load and lags if you
|
can access a country- or organization-local mirror of opensource projects.
|
|
Component build
|
|
If you are only working on a single component, you can just build it using
|
following:
|
|
setup the workspace for building components
|
|
$ cd (your-workspace)/components ; gmake setup
|
|
build the individual component
|
|
$ cd (component-dir) ; gmake publish
|
|
Complete Top Down build
|
|
Complete top down builds are also possible by simply running
|
|
$ cd (your-workspace)/components
|
$ gmake publish
|
|
The 'publish' target will build each component and publish it to the
|
workspace IPS repo.
|
|
Tools to help facilitate build zone creation will be integrated
|
shortly. If the zone you create to build your workspace in does not have
|
networking enabled, you can pre-download any community source archives into
|
your workspace from the global with:
|
|
$ cd (your-workspace)/components
|
$ gmake download
|
|
You can add parallelism to your builds by adding '-j (jobs)' to your gmake
|
command line arguments. Note that if the host is constrained on resources
|
or the component source Makefiles are poorly thought out, parallel builds
|
can fail - in this case subsequent single-job (sequential) builds should
|
succeed to complete the missing build products.
|
|
It is worth noting that the OpenIndiana Hipster build server uses the
|
'COMPONENT_BUILD_ARGS=-j4' option by default for moderate parallelization
|
of its builds.
|
|
The gate should only incrementally build what it needs to based on what has
|
changed since you last built it.
|