Item1944: Debian autobuilder version independence, build isolation and build from source packages

I've been working on a .deb autobuilder based on BuildContrib. The new builder separates repository update from the actual build so the build and all repository-updated scripts can run in an isolated virtual machine. The package build is then rerun to make sure the source packages successfully build the package (rather than building debs directly from the repository or the tarball). This allows the packages to be current rather than waiting for a new 1.0.x release and tarball.

The repository of autobuilt packages is at http://debmarshal.debian.net/foswiki/ . The pubkey used to sign the repository is in http://debmarshal.debian.net/ The current /etc/apt/sources.list line to pull from this repository is:

deb http://debmarshal.debian.net/foswiki foswiki-1.0/latest local

I'll attached the proposed 1.0 branch build scripts in a moment. The trunk release branch is not currently building successfully, and will likely need additional tweaks, so both release tracks currently have the same 1.0.6 packages in them.

No extensions are being built yet. Extensive dependency analysis is being done on the packages.

• host-1.0.sh: Runs on the host machine outside of the repository

• build-1.0.sh: Builds the packages. Run the version in the repository.

---

mmm, first thought is that this is similar to the bits I've built, but then I realized that this is more scary than debian experimental - what's in svn has had no real testing, and in the case of the extensions, is often partial, under development, sometimes not even functional.

there's a nightly build tool in the core/tools dir, which at least makes an effort to test things before it just makes a tarball, so I'd suggest you run that rather than just a blind tar - that way we all benefit from improvements to that script (its used on my nightly test&build systems)

-- SvenDowideit - 20 Aug 2009

---

I can add running the automatic unit tests to the build script, so any failing test will fail to produce a .deb at all. I had intended to do that next after the basic builder was working, but can do it now if that helps answer the safety-of-the-debs, vs the safety-of-the-build issue that was frontmost in my mind (since it is pulling code out of a subversion repository and running it automatically).

Some background on where I'm going with this: I'd like to converge the Foswiki source release with the Debian release, so when foswiki has an official release, there are .debs of it right away, instead of beginning work on packaging right then (and either shipping an old Foswiki or not shipping at all). That maximizes the amount of beating-on time the .debs can get before they are sent to Debian/unstable, and makes them as current with the foswiki source as possible. The next Debian release freeze is only a few months away now (I know, shocking), which might line up well with a 1.1.0 Foswiki release if they all arrive at once. If 1.1.0 isn't ready though, these two scripts though are autobuilding the 1.0.x release branch today, and should automatically cope with a 1.0.7 release when/if it occurs.

There will likely be a lot of breakage of both the source and the packaging as noted, but occasionally both will work together, and those working .debs are being captured as input to further testing. This way none use the same filenames, so all can coexist in the same repository. The "latest" symlink just points to the most recent successful build, other symlinks can be pointed at builds known to pass other levels of testing/canarying.

It isn't my intention to autoupload any of these directly to Debian (which wouldn't be allowed in any event), but to stress out these builds, and then rebuild a good one from source, modify its release number to make it official, and upload that source+binary package to Debian.

Some of the problems I ran into with the existing build required changes to get this far, and by the time I'd made them all there wasn't anything in common. And as it is pulling from the repository, any changes I did make would have been immediately wiped out by unchanged foswiki versions - so the filenames, for development, had to be different anyway.

-- DrakeDiedrich - 24 Aug 2009

This was modified and uploaded for 1.0.7 and trunk, and is currently running at http://debmarshal.debian.net/, producing .debs of core for 1.0.x and trunk.

-- DrakeDiedrich - 23 Oct 2009