diff options
Diffstat (limited to 'share/man/man7/development.7')
| -rw-r--r-- | share/man/man7/development.7 | 698 |
1 files changed, 698 insertions, 0 deletions
diff --git a/share/man/man7/development.7 b/share/man/man7/development.7 new file mode 100644 index 000000000000..af8db3a75d21 --- /dev/null +++ b/share/man/man7/development.7 @@ -0,0 +1,698 @@ +.\" Copyright (C) 1998 Matthew Dillon. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd December 21, 2002 +.Dt DEVELOPMENT 7 +.Os +.Sh NAME +.Nm development +.Nd "introduction to development with the FreeBSD codebase" +.Sh DESCRIPTION +This manual page describes how an ordinary system operator, +.Ux +administrator, or developer +can, without any special permission, obtain, maintain, and modify the +.Fx +codebase as well as how to maintain a master build which can +then be exported to other machines in your network. +This manual page +is targeted to system operators, programmers, and developers. +.Pp +Please note that what is being described here is based on a complete +.Fx +environment, not just the +.Fx +kernel. +The methods described +here are as applicable to production installations as it is to development +environments. +You need a good 12-17GB of disk space on one machine to make this work +conveniently. +.Sh SETTING UP THE ENVIRONMENT ON THE MASTER SERVER +Your master server should always run a stable, production version of the +.Fx +operating system. +This does not prevent you from doing -CURRENT +builds or development. +The last thing you want to do is to run an +unstable environment on your master server which could lead to a situation +where you lose the environment and/or cannot recover from a mistake. +.Pp +Create a huge partition called +.Pa /FreeBSD . +8-12GB is recommended. +This partition will contain nearly all the development environment, +including the CVS tree, broken-out source, and possibly even object files. +You are going to export this partition to your other machines via a +READ-ONLY NFS export so do not mix it with other more security-sensitive +partitions. +.Pp +You have to make a choice in regards to +.Pa /usr/obj . +You can put +.Pa /usr/obj +in +.Pa /FreeBSD +or you can make +.Pa /usr/obj +its own partition. +I recommend making it a separate partition for several reasons. +First, +as a safety measure since this partition is written to a great deal. +Second, because you typically do not have to back it up. +Third, because it makes it far easier to mix and match the development +environments which are described later in this document. +I recommend a +.Pa /usr/obj +partition of at least 5GB. +.Pp +On the master server, use +.Xr cvsup 1 Pq Pa ports/net/cvsup +to automatically pull down and maintain +the +.Fx +CVS archive once a day. +The first pull will take a long time, +it is several gigabytes, but once you have it, +the daily synchronizations will be quite small. +.Bd -literal -offset 4n +mkdir /FreeBSD/FreeBSD-CVS +rm -rf /home/ncvs +ln -s /FreeBSD/FreeBSD-CVS /home/ncvs +.Ed +.Pp +The +.Xr cron 8 +job should look something like this (please randomize the time of +day!). +Note that you can use the +.Xr cvsup 1 +configuration file example directly from +.Pa /usr/share/examples +without modification by supplying appropriate arguments +to +.Xr cvsup 1 . +.Bd -literal -offset 4n +33 6 * * * /usr/local/bin/cvsup -g -r 20 -L 2 -h cvsup.freebsd.org /usr/share/examples/cvsup/cvs-supfile +.Ed +.Pp +Run the +.Xr cvsup 1 +manually the first time to pull down the archive. +It could take +all day depending on how fast your connection is! +You will run all +.Xr cvsup 1 +and +.Xr cvs 1 +operations as +.Dq Li root +and you need to set up a +.Pa ~/.cvsrc +.Pq Pa /root/.cvsrc +file, as shown below, for proper +.Xr cvs 1 +operation. +Using +.Pa ~/.cvsrc +to specify +.Xr cvs 1 +defaults is an excellent way to +.Dq "file and forget" , +but you should never forget that you put them in there. +.Bd -literal -offset 4n +# cvs -q +diff -u +update -Pd +checkout -P +.Ed +.Pp +Now use +.Xr cvs 1 +to check out a -STABLE source tree and a -CURRENT source tree, +as well as ports and docs, to create your initial source environment. +Keeping the broken-out source and ports in +.Pa /FreeBSD +allows you to export +it to other machines via read-only NFS. +This also means you only need to edit/maintain files in one place and all +your clients automatically pick up the changes. +.Bd -literal -offset 4n +mkdir /FreeBSD/FreeBSD-4.x +mkdir /FreeBSD/FreeBSD-current + +cd /FreeBSD/FreeBSD-4.x +cvs -d /home/ncvs checkout -rRELENG_4 src + +cd /FreeBSD/FreeBSD-current +cvs -d /home/ncvs checkout src +cvs -d /home/ncvs checkout ports +cvs -d /home/ncvs checkout doc +.Ed +.Pp +Now create a softlink for +.Pa /usr/src +and +.Pa /usr/src2 . +On the main server I always point +.Pa /usr/src +at -STABLE and +.Pa /usr/src2 +at -CURRENT. +On client machines I usually do not have a +.Pa /usr/src2 +and I make +.Pa /usr/src +point at whatever version of +.Fx +the client box is intended to +run. +.Bd -literal -offset 4n +cd /usr +rm -rf src src2 +ln -s /FreeBSD/FreeBSD-4.x/src src (could be -CURRENT on a client) +ln -s /FreeBSD/FreeBSD-current/src src2 (MASTER SERVER ONLY) +.Ed +.Pp +Now you have to make a choice for +.Pa /usr/obj . +Well, hopefully you made it already and chose the partition method. +If you +chose poorly you probably intend to put it in +.Pa /FreeBSD +and, if so, this is +what you want to do: +.Bd -literal -offset 4n +(ONLY IF YOU MADE A POOR CHOICE AND PUT /usr/obj in /FreeBSD!) +mkdir /FreeBSD/obj +cd /usr +rm -rf obj +ln -s /FreeBSD/obj obj +.Ed +.Pp +Alternatively you may chose simply to leave +.Pa /usr/obj +in +.Pa /usr . +If your +.Pa /usr +is large enough this will work, but I do not recommend it for +safety reasons +.Pa ( /usr/obj +is constantly being modified, +.Pa /usr +is not). +.Pp +Note that exporting +.Pa /usr/obj +via read-only NFS to your other boxes will +allow you to build on your main server and install from your other boxes. +If you also want to do builds on some or all of the clients you can simply +have +.Pa /usr/obj +be a local directory on those clients. +You should never export +.Pa /usr/obj +read-write, it will lead to all sorts of +problems and issues down the line and presents a security problem as well. +It is far easier to do builds on the master server and then only do installs +on the clients. +.Pp +I usually maintain my ports tree via CVS. +It is sitting right there in the master CVS archive and I have even told you +to check it out (see above). +With some fancy softlinks you can make the ports tree available both on your +master server and on all of your other machines. +Note that the ports tree exists only on the HEAD CVS branch, so its always +-CURRENT even on a -STABLE box. +This is what you do: +.Bd -literal -offset 4n +(THESE COMMANDS ON THE MASTER SERVER AND ON ALL CLIENTS) +cd /usr +rm -rf ports +ln -s /FreeBSD/FreeBSD-current/ports ports + +cd /usr/ports (this pushes into the softlink) +rm -rf distfiles (ON MASTER SERVER ONLY) +ln -s /usr/ports.distfiles distfiles (ON MASTER SERVER ONLY) + +mkdir /usr/ports.distfiles +mkdir /usr/ports.workdir +.Ed +.Pp +Since +.Pa /usr/ports +is softlinked into what will be read-only on all of your +clients, you have to tell the ports system to use a different working +directory to hold ports builds. +You want to add a line to your +.Xr make.conf 5 +file on the master server +and on all your clients: +.Bd -literal -offset 4n +WRKDIRPREFIX=/usr/ports.workdir +.Ed +.Pp +You should try to make the directory you use for the ports working directory +as well as the directory used to hold distfiles consistent across all of your +machines. +If there is not enough room in +.Pa /usr/ports.distfiles +and +.Pa /usr/ports.workdir +I usually make those softlinks (since this is on +.Pa /usr +these are per-machine) to +where the distfiles and working space really are. +.Sh EXPORTING VIA NFS FROM THE MASTER SERVER +The master server needs to export +.Pa /FreeBSD +and +.Pa /usr/obj +via NFS so all the +rest of your machines can get at them. +I strongly recommend using a read-only export for both security and safety. +The environment I am describing in this manual page is designed primarily +around read-only NFS exports. +Your exports file on the master server should contain the following lines: +.Bd -literal -offset 4n +/FreeBSD -ro -alldirs -maproot=root: -network YOURLAN -mask YOURLANMASK +/usr/obj -ro -alldirs -maproot=root: -network YOURLAN -mask YOURLANMASK +.Ed +.Pp +Of course, NFS server operations must also be configured on that machine. +This is typically done via your +.Pa /etc/rc.conf : +.Bd -literal -offset 4n +nfs_server_enable="YES" +nfs_server_flags="-u -t -n 4" +.Ed +.Sh THE CLIENT ENVIRONMENT +All of your client machines can import the development/build environment +directory simply by NFS mounting +.Pa /FreeBSD +and +.Pa /usr/obj +from the master server. +A typical +.Pa /etc/fstab +entry on your client machines will be something like this: +.Bd -literal -offset 4n +masterserver:/FreeBSD /FreeBSD nfs ro,bg 0 0 +masterserver:/usr/obj /usr/obj nfs ro,bg 0 0 +.Ed +.Pp +And, of course, you should configure the client for NFS client operations +via +.Pa /etc/rc.conf . +In particular, this will turn on +.Xr nfsiod 8 +which will improve client-side NFS +performance: +.Bd -literal -offset 4n +nfs_client_enable="YES" +.Ed +.Pp +Each client should create softlinks for +.Pa /usr/ports +and +.Pa /usr/src +that point +into the NFS-mounted environment. +If a particular client is running -CURRENT, +.Pa /usr/src +should be a softlink to +.Pa /FreeBSD/FreeBSD-current/src . +If it is running -STABLE, +.Pa /usr/src +should be a softlink to +.Pa /FreeBSD/FreeBSD-4.x/src . +I do not usually create a +.Pa /usr/src2 +softlink on +clients, that is used as a convenient shortcut when working on the source +code on the master server only and could create massive confusion (of the +human variety) on a client. +.Bd -literal -offset 4n +(ON EACH CLIENT) +cd /usr +rm -rf ports src +ln -s /FreeBSD/FreeBSD-current/ports ports +ln -s /FreeBSD/FreeBSD-XXX/src src +.Ed +.Pp +Do not forget to create the working directories so you can build ports, as +previously described. +If these are not good locations, make them softlinks to the correct location. +Remember that +.Pa /usr/ports/distfiles +is exported by +the master server and is therefore going to point to the same place +(typically +.Pa /usr/ports.distfiles ) +on every machine. +.Bd -literal -offset 4n +mkdir /usr/ports.distfiles +mkdir /usr/ports.workdir +.Ed +.Sh BUILDING KERNELS +Here is how you build a -STABLE kernel (on your main development box). +If you want to create a custom kernel, copy +.Pa GENERIC +to +.Pa KERNELNAME +and then edit it before configuring and building. +The kernel configuration file lives in +.Pa /usr/src/sys/i386/conf/KERNELNAME . +.Bd -literal -offset 4n +cd /usr/src +make buildkernel KERNCONF=KERNELNAME +.Ed +.Pp +.Sy WARNING! +If you are familiar with the old config/cd/make method of building +a -STABLE kernel, note that the +.Xr config 8 +method will put the build environment in +.Pa /usr/src/sys/i386/compile/KERNELNAME +instead of in +.Pa /usr/obj . +.Pp +Building a -CURRENT kernel +.Bd -literal -offset 4n +cd /usr/src2 (on the master server) +make buildkernel KERNCONF=KERNELNAME +.Ed +.Sh INSTALLING KERNELS +Installing a -STABLE kernel (typically done on a client, +only do this on your main development server if you want to install a new +kernel for your main development server): +.Bd -literal -offset 4n +cd /usr/src +make installkernel KERNCONF=KERNELNAME +.Ed +.Pp +If you are using the older config/cd/make build mechanism for -STABLE, you +would install using: +.Bd -literal -offset 4n +cd /usr/src/sys/i386/compile/KERNELNAME +make install +.Ed +.Pp +Installing a -CURRENT kernel (typically done only on a client) +.Bd -literal -offset 4n +(remember /usr/src is pointing to the client's specific environment) +cd /usr/src +make installkernel KERNCONF=KERNELNAME +.Ed +.Sh BUILDING THE WORLD +This environment is designed such that you do all builds on the master server, +and then install from each client. +You can do builds on a client only if +.Pa /usr/obj +is local to that client. +Building the world is easy: +.Bd -literal -offset 4n +cd /usr/src +make buildworld +.Ed +.Pp +If you are on the master server you are running in a -STABLE environment, but +that does not prevent you from building the -CURRENT world. +Just +.Xr cd 1 +into the appropriate source directory and you are set. +Do not +accidentally install it on your master server though! +.Bd -literal -offset 4n +cd /usr/src2 +make buildworld +.Ed +.Sh INSTALLING THE WORLD +You can build on your main development server and install on clients. +The main development server must export +.Pa /FreeBSD +and +.Pa /usr/obj +via read-only NFS to the clients. +.Pp +.Em NOTE!!! +If +.Pa /usr/obj +is a softlink on the master server, it must also be the EXACT +SAME softlink on each client. +If +.Pa /usr/obj +is a directory in +.Pa /usr +or a mount point on the master server, +then it must be (interchangeably) a directory in +.Pa /usr +or a mount point on +each client. +This is because the +absolute paths are expected to be the same when building the world as when +installing it, and you generally build it on your main development box +and install it from a client. +If you do not set up +.Pa /usr/obj +properly you will not be able to build on +machine and install on another. +.Bd -literal -offset 4n +(ON THE CLIENT) +(remember /usr/src is pointing to the client's specific environment) +cd /usr/src +make installworld +.Ed +.Pp +.Sy WARNING! +If builds work on the master server but installs do not work from the +clients, for example you try to install and the client complains that +the install tried to write into the read-only +.Pa /usr/obj , +then it is likely +that the +.Xr make.conf 5 +file on the client does not match the one on the +master server closely enough and the install is trying to install something +that was not built. +.Sh DOING DEVELOPMENT ON A CLIENT (NOT JUST INSTALLING) +Developers often want to run buildkernel's or buildworld's on client +boxes simply to life-test the box. +You do this in the same manner that you buildkernel and buildworld on your +master server. +All you have to do is make sure that +.Pa /usr/obj +is pointing to local storage. +If you followed my advise and made +.Pa /usr/obj +its own partition on the master +server, +then it is typically going to be an NFS mount on the client. +Simply unmounting +.Pa /usr/obj +will leave you with a +.Pa /usr/obj +that is a +subdirectory in +.Pa /usr +which is typically local to the client. +You can then do builds to your heart's content! +.Sh MAINTAINING A LOCAL BRANCH +I have described how to maintain two versions of the source tree, a stable +version in +.Pa /FreeBSD/FreeBSD-4.x +and a current version in +.Pa /FreeBSD/FreeBSD-current . +There is absolutely nothing preventing you +from breaking out other versions of the source tree +into +.Pa /FreeBSD/XXX . +In fact, my +.Pa /FreeBSD +partition also contains +.Ox , +.Nx , +and various flavors of +.Tn Linux . +You may not necessarily be able to build +.Pf non- Fx +operating systems on +your master server, but being able +to collect and manage source distributions from a central server is a very +useful thing to be able to do and you can certainly export to machines +which can build those other operating systems. +.Pp +Many developers choose to maintain a local branch of +.Fx +to test patches or build a custom distribution. +This can be done with CVS or another source code management system +(SubVersion, Perforce, BitKeeper) with its own repository. +Since the main +.Fx +tree is based on CVS, the former is convenient. +.Pp +First, you need to modify your +.Xr cvsup 1 +environment to avoid it modifying +the local changes you have committed to the repository. +It is important to remove the +.Ic delete +keyword from your +.Pa supfile +and to add the +.Pa CVSROOT +subdirectory to your +.Pa refuse +file. +For more information, see +.Xr cvsup 1 . +.Pp +The +.Fx +version of +.Xr cvs 1 +examines a custom environmental variable, +.Ev CVS_LOCAL_BRANCH_NUM , +which specifies an integer to use when doing a +.Xr cvs 1 +.Cm tag Ns / Ns Cm rtag . +Set this number to something high (say 1000) to avoid colliding +with potential future branches of the main repository. +For example, +branching a file with version 1.4 produces 1.4.1000. +Future commits to this branch will produce revisions 1.4.1000.1, +1.4.1000.2, etc. +.Pp +To fork your local branch, do: +.Bd -literal -offset 4n +cvs rtag -r RELENG_4 -b LOCAL_RELENG_4 src +.Ed +.Pp +After this, you can check out a copy from your local repository using the +new tag and begin making changes and committing them. +For more information on using CVS, see +.Xr cvs 1 . +.Pp +.Sy WARNING! +The +.Xr cvsup 1 +utility may blow away changes made on a local branch in +some situations. +This has been reported to occur when the master CVS repository is +directly manipulated or an RCS file is changed. +At this point, +.Xr cvsup 1 +notices that the client and server have entirely +different RCS files, so it does a full replace instead of trying to +send just deltas. +Ideally this situation should never arise, but in the real world it +happens all the time. +.Pp +While this is the only scenario where the problem should crop up, +there have been some suspicious-sounding reports of +.Ev CVS_LOCAL_BRANCH_NUM +lossage that cannot be explained by this alone. +Bottom line is, if you value your local branch then you +should back it up before every update. +.Sh UPDATING VIA CVS +The advantage of using +.Xr cvsup 1 +to maintain an updated copy of the CVS +repository instead of using it to maintain source trees directly is that you +can then pick and choose when you bring your source tree (or pieces of your +source tree) up to date. +By using a +.Xr cron 8 +job to maintain an updated CVS repository, you can update +your source tree at any time without any network cost as follows: +.Bd -literal -offset 4n +(on the main development server) +cd /usr/src +cvs -d /home/ncvs update +cd /usr/src2 +cvs -d /home/ncvs update +cd /usr/ports +cvs -d /home/ncvs update +.Ed +.Pp +It is that simple, and since you are exporting the whole lot to your +clients, your clients have immediate visibility into the updated +source. +This is a good time to also remind you that most of the +.Xr cvs 1 +operations you do will be done as +.Dq Li root , +and that certain options are +required for CVS to operate properly on the +.Fx +repository. +For example, +.Fl Pd +is necessary when running +.Nm cvs Cm update . +These options are typically placed in your +.Pa ~/.cvsrc +(as already described) +so you do not have to re-specify them every time you run a +.Xr cvs 1 +command. +Maintaining the CVS repository also gives you far more flexibility +in regards to breaking out multiple versions of the source tree. +It is a good idea to give your +.Pa /FreeBSD +partition a lot of space (I recommend +8-12GB) precisely for that reason. +If you can make it 15GB I would do it. +.Pp +I generally do not +.Nm cvs Cm update +via a +.Xr cron 8 +job. +This is because I generally want the source to not change out from under me +when I am developing code. +Instead I manually update the source every so often...\& when I feel it is +a good time. +My recommendation is to only keep the CVS repository synchronized via +.Xr cron 8 . +.Sh SEE ALSO +.Xr crontab 1 , +.Xr crontab 5 , +.Xr make.conf 5 , +.Xr build 7 , +.Xr firewall 7 , +.Xr release 7 , +.Xr tuning 7 , +.Xr diskless 8 +.Sh HISTORY +The +.Nm +manual page was originally written by +.An Matthew Dillon Aq dillon@FreeBSD.org +and first appeared +in +.Fx 5.0 , +December 2002. |