20210310
It took me about an hour to make a PKGBUILD for my simple, non-compiled piece of software to be published on AUR. In contrast, it took me a few days to figure out how to build suitable .deb packages for publishing in a PPA on Launchpad. In this post, I’ll try to describe some of the initial pain points of mine.
The Debian package format is really old, and it shows. There’s a billion of metadata files to take care of, and barely any suitable tutorials for beginners. At best, you’ll learn how to build binary packages, not suitable for publishing in a PPA (which only accept source packages and builds the binaries itself).
First, you need to realize that there are source packages and binary packages.
Binary packages are the .deb files that actually contain the software.
A source package is, confusingly, multiple files, and you need to submit them
all to Launchpad.
You can distribute binary packages directly to your users, but they would have
to fetch & install the new version manually every time there’s an upgrade.
If you could set up a repository and just point the users to it, they would get
the new versions naturally via the package manager (apt).
Canonical’s Launchpad provides a very handy PPA (Personal Package Archive)
service so that anyone can set up a repository.
Users could then use add-apt-repository ppa:... and get the packages in a
standard and convenient way.
There’s a myriad of tools to build and maintain Debian packages. The Debian New Maintainers’ Guide provides a short summary of how these tools interact. This tutorial assumes that your software lives in a Git repository and you’d like to use Git to maintain the packaging metadata in the same repository. This process is greatly aided by the git-buildpackage tool. We still need to install a bunch of other stuff though; the complete command line to install the required tools would be something like
Many of the tools pick up particular metadata (like the maintainer name and email address) from environment variables. You can put something like
export DEBFULLNAME='John Doe'
export DEBEMAIL='John.Doe@example.com'
in your .bashrc to set them globally.
Let’s create a repository to try things out. It’ll contain a single executable shell script test.sh, which only outputs the string “test”.
This is going to be version 1.0 of our project, let’s tag it as such.
All of the Debian packaging tools are tailored to the following use-case.
This disconnect means that maintaining the Debian packaging files in the
master branch is inconvenient using the existing tools.
At the very least, you should create a separate branch for doing packaging
work.
In addition, Debian (and hence, Ubuntu) is not a rolling-release distribution. That means that there’re regular releases, and the software version shouldn’t change too much during a lifetime of a single release. Once Debian makes a release, the software version is more or less fixed, and security fixes from future versions should be backported separately for each of the supported Debian/Ubuntu releases.
Except there is a rolling-release distribution of Debian, and it’s called “unstable” or “sid”. The bleeding-edge packaging work should target the “unstable” distribution.
So, let’s create a new branch debian for our packaging work:
All the packaging tools assume there’s a separate folder “debian” that contains
the package metadata files.
There’s a handy tool dh_make that creates the directory and populates it with
a number of template metadata files.
Using it is not so simple though.
First of all, it assumes that there’s a properly named tarball with the project
sources available in the parent directory.
Why?
Who knows.
Let’s create said tarball:
The tarball name should follow the NAME_VERSION.orig.tar.gz pattern exactly!
Anyway, now is the time to run dh_make:
I’m using the MIT License for our little script, hence the --copyright mit
argument.
In addition, every package in Debian is either “single”, “arch-independent”,
“library” or “python”.
I’m not sure what the exact differences between those are, but a shell script
is clearly CPU architecture-independent, hence the --indep argument.
If it was a compiled executable, it would be a “single” (--single) package.
dh_make created the “debian” directory for us, filled with all kinds of
files.
The only required ones are “changelog”, “control”, “source”, “rules” and the
“source” directory.
Let’s remove every other file for now:
You can study the exact format of the metadata files in the Debian New Maintainers’ Guide, but for now let’s keep it simple:
test (1.0-1) unstable; urgency=medium
* Initial release.
-- John Doe <John.Doe@example.com> Wed, 10 Mar 2021 16:15:19 +0000
Source: test
Section: utils
Priority: optional
Maintainer: John Doe <John.Doe@example.com>
Build-Depends: debhelper-compat (= 12)
Standards-Version: 4.4.1
Homepage: https://example.com/test
Package: test
Architecture: all
Depends: ${misc:Depends}
Description: This is a test package.
This is a test package, just trying out Debian packaging.
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: test
Upstream-Contact: John Doe <John.Doe@example.com>
Source: https://example.com/test
Files: *
Copyright: 2021 John Doe <John.Doe@example.com>
License: MIT
Files: debian/*
Copyright: 2021 John Doe <John.Doe@example.com>
License: MIT
License: MIT
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
.
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
#!/usr/bin/make -f
%:
dh $@
The “control” and “copyright” files are fairly straighforward.
The “changelog” file has a strict format and is supposed to be maintained using
the dch tool (luckily, git-buildpackage helps with that; more on that later).
The “rules” file is an executable Makefile, and actually controls how the
software is built.
Building a package involves invoking many predefined targets in this Makefile;
for now, we’ll resort to delegating everything to the dh tool.
It’s the Debhelper tool; it’s a magic set of scripts that contain an
unbelievable amount of hidden logic that’s supposed to aid package maintainers
in building the software.
For example, if the package is supposed to be built using the standard
./configure && make && make install sequence, it’ll do this automatically.
If it’s a Python package with setup.py, it’ll use the Python package-building
utilities, etc.
We don’t want any of that, we just want to copy test.sh to /usr/bin.
It can be taken care of using the dh_install script.
While building the package, it’ll get executed by dh, read the
“debian/test.install” file and copy the files listed there to the specified
directories.
Our test.install should look like this:
test.sh usr/bin
At this point, we can actually build a proper Debian package!
This command will generate a bunch of files in the parent directory.
The one of interest to us is “test_1.0-1_all.deb”.
We can install it using dpkg:
We can now execute test.sh, and it’ll hopefully print the string “test”.
This .deb file can be distributed to other users, but is no good for uploading to Launchpad. For one, it’s a binary package, and we need source packages for Launchpad to build itself. Second, it’s unsigned, which is also a no-no.
I’m not going to describe how to set up a GnuPG key and upload it to the Ubuntu keyserver (keyserver.ubuntu.com), but it’s pretty straightforward once you know the basics of GnuPG key handling.
One disadvantage of the dpkg-buildpackage tool is that it creates a lot of
files in the “debian” directory; their purpose is unclear to me.
For now, you can delete them, leaving only the original “changelog”, “control”,
“copyright”, “rules”, “test.install” and the “source” directory.
git-buildpackage is a wonderful tool that helps with managing the packaging
work in the upstream repository.
Please refer to its manual to learn how to use it properly.
We need to configure it so that it knows how the release tags look like
(vVERSION), how the packaging branch is called (debian) and where to put
the generated files.
Create “debian/gbp.conf” with the following contents:
[DEFAULT]
upstream-tag = v%(version)s
debian-branch = debian
pristine-tar = False
export-dir = ../build-area/
One unclear line here is pristine-tar = False.
It turns out, a lot of Debian package maintainers use the pristine-tar tool
to create “pristine”, byte-for-byte reproducible tarballs of the upstream
software.
This is just more headache for us, so we’re not going to use that;
git-buildpackage will just use the normal git archive to create tarballs.
First, commit the packaging work we just made:
We can now build the package using git-buildpackage:
The tool will try to sign the packages, so this assumes that you have your GnuPG key set up!
If all went right, it just built the packages in the ../build-area directory.
And it hasn’t crapped all over the working directory too!
Similar to dpkg-buildpackage, it builds binary packages by default.
To build source packages, it needs to be invoked with the -S argument:
It’ll build the source package in the same directory (you’ll notice a lot of files having the “_source” suffix). If all is well, we can tag the packaging work we’ve just completed:
This will create the debian/1.0-1 tag in the repository.
We are now ready to upload the source package to Launchpad.
It’s done using the dput tool.
The naive way would fail:
This is due to the fact that we’ve specified that we’re targetting the “unstable” distribution in debian/changelog. There’s no “unstable” distribution of Ubuntu though; we need to manually specify the minimal-supported version (e.g. “bionic”):
What about other distributions? Well, if the binary package doesn’t need recompiling, we can use Launchpad’s “Copy packages” feature; this is well-described in this Ask Ubuntu question.
When a new version is released, git-buildpackage helps to integrate it to the
packaging branch.
Let’s say the new version is tagged v1.1:
The above command will update debian/changelog; modify it manually to target the usual “unstable” distribution instead of “UNRELEASED” and update the version to something like “1.1-1”.
This will build the source package for the new version in the ../build-area directory; you can then upload it Launchpad and copy the built binary packages.
This fucking sucks. What’s the way to sanely manage the repository if the build/runtime dependencies are different for different Ubuntu versions? I have no idea. Some pointers to help you understand what’s going on in this tutorial more deeply:
Good luck with this, because I’m definitely overwhelmed.
My blog. Feel free to contribute or contact me.