Things you should double check twice (or more) before releasing your software

When you start packaging more and more things for your distribution, you realize how things would be easier if upstream was slighly more concerned about how distributions work. Here are a couple examples of things you shall double check before releasing your work.

1/ Don’t install headers with your runtime stuff.

This sounds obvious, right ? Runtime applications should not be used as a build dependency for anything or it’s no longer a runtime thing.

A bad example: kde-runtime
A good example: nepomuk-core

# ls /kde/inst/kde-runtime/include

knotifyconfig.h knotify_export.h knotifyplugin.h

Now, sit and think: How can you use these headers ?
Simple answer, you can’t because they’re installed by a runtime module, which means:

– There’s no CMake config files installed,
– There’s consequently no way to know these headers locations nor how to link to knotify, anyway.

A solution? Don’t installs headers in a runtime-only module or move your code elsewhere.

That’s the reason why a nepomuk-core module was created before the KDE 4.9 release. We then could get rid of all the nepomuk-dms copies in different KDE projects.

2/ Applications icons *MUST* be installed in the hicolor namespace

Bad examples: lots of KDE applications.
A (very) good example: Okular

Because KDE apps are great, they should also look great in non-KDE environments.

Again, let’s sit and think:

myApp.desktop has an Icon=myIcon line and myIcon.png is only installed in the Oxygen namespace.
Now, what happens if a Gnome, XFCE or $whatever_DE_using_a_different_icon_theme user installs myApp?

Simple, the application icon won’t be visible. Despite what was discussed here, the Oxygen icons set shouldn’t be a must-have to use KDE applications.

Quoting the Icon theme spec:

« So, you’re an application author, and want to install application icons so that they work in the KDE and Gnome menus. Minimally you should install a 48×48 icon in the hicolor theme. This means installing a PNG file in $prefix/share/icons/hicolor/48×48/apps. Optionally you can install icons in different sizes. For example, installing a svg icon in $prefix/share/icons/hicolor/scalable/apps means most desktops will have one icon that works for all sizes. You might even want to install icons with a look that matches other well known themes so your application will fit in with some specific desktop environment.

It is recommended that the icons installed in the hicolor theme look neutral, since it is a fallback theme that will be used in combination with some very different looking themes. But if you don’t have any neutral icon, please install whatever icon you have in the hicolor theme so that all applications get at least some icon in all themes.»

(Source)

After reading this, you should also have realized something: Icons/pixmaps used in your applications that do not exist in the hicolor theme should be installed by your application (duplication is not an issue since these icons won’t be installed in the icons theme directory).

Additional note on this: in openSUSE, rpmlint now throws an error if the application icon is defined in a desktop file but is not installed.

3/ Don’t include half the world in your tarball.

While this may be convenient for your build system, this is a maintainance nightmare for packagers and takes hours to rebuild everything when we have to change even a tiny line.

(very) Bad example: Digikam

Let’s examine the Digikam 2.9.0 tarball: its size is ~55 Mb. Once uncompressed, 29% of the data come from the “core” subdir and 25% come from the “extra” one.

Now, let’s see what in this extra dir:

/tmp/digikam-2.9.0 # ls extra
CMakeLists.txt kipi-plugins libkdcraw libkexiv2 libkface libkgeomap libkipi libksane libkvkontakte libmediawiki

Why is that so bad ?

– kipi-plugins can be used at runtime by Gwenview but didn’t get a proper release since ages (it’s nowhere on ftp.kde.org and the last release on sourceforge is more than one year old),
– libkvkontakte is a build requirement for the akonadi-vkontakte resource, but was never released),
– libksane is already shipped with kdegraphics,
etc…

From a packager point of view, this raises several issues:
– We have to split the packages in sub-packages that can clash when they’re also shipped with KDE,
– We usually don’t want to use embedded 3rd party libraries and prefer some proper releases.
– And more important, a couple libraries that are used to develop other applications don’t have a release cycle.

4/ Embedded 3rd party stuff should not be mandatory

Bad example: qt-creator.

Instead of requiring Botan at build time, qt-creator ships a copy. It could be easily patched out before the 2.6 pre-releases in order to use the distribution Botan. With qt-creator 2.6, it suddenly required more work:

– Commit fcc9ba4 not only updated the botan version, it also merged everything in an ugly blob.

While packaging qt-creator 2.6 beta, some includes had to be added in several places to build with the system botan instead of this blob.

You may think it can’t be worse for packagers? I’m sorry to disappoint you…

– Commit 6f7ce3f4 added a Qt specific patch to the 3rd party embedded Botan. So, now, we not only have to patch qt-creator in order to use the distro Botan, we also have to revert a commit in order to make qt-creator build.

Really, if you can’t resist the temptation to add 3rd party applications in your tarballs, at least:

– make sure to add an option to disable it,

– don’t alter the 3rd party code… and even in this case…

5/ Make sure your application still builds if embedded 3rd party libraries can be disabled

Bad example: Qt5

Qt5 provides a configure option to use the system pcre library. Nothing really new, Qt4 also has the -system-pcre option.

However, for some unknown reason, the system pcre wasn’t found. Using the -v option gives some clues:

PCRE auto-detection… ()
g++ -c -pipe -O2 -Wall -W -fPIE -I/usr/include -I../../../mkspecs/linux-g++ -I. -I/usr/include -I/usr/include/pgsql -I/usr/include/pgsql/server -I/usr/include/mysql -o pcre.o pcre.cpp
g++ -Wl,-O1 -o pcre pcre.o -L/usr/lib64 -lpcre16
/usr/lib64/gcc/x86_64-suse-linux/4.7/../../../../x86_64-suse-linux/bin/ld: cannot find -lpcre16
collect2: error: ld returned 1 exit status
gmake: *** [pcre] Error 1
PCRE disabled.

You may wonder where this -lpcre16 comes from and you will likely not find a reason. It’s just hardcoded in qtbase/config.tests/unix/pcre/pcre.pro.
It doesn’t matter whether the libpcre pkgconfig file mentions Libs: -L${libdir} -lpcre, this file is just ignored.

*EDIT 2012-11-07* This issue comes from a lack of documentation when using  ./configure –help.  -system-pcre means… “Check whether pcre was built with –enable-pcre16”.
The 7th point could be: “Packagers are not seers, don’t forget docs or build howtos.”

6/ Make sure your application runs if installed in a custom prefix (iow: don’t assume everyone has the same setup as you)
Bad example: plasma (more precisely, libplasma, so this also breaks kactivities)

A couple months ago, Jenkins appeared in the KDE infrastructure. It’s quite nice to detect missing dependencies and includes here and there.

The secret: Jenkins installs everything in a different prefix, updates its environment variables and tries the next target. But what would happen if you try to run KDE with such a setup ?

I can answer: almost all works fine. All… but Plasma. This is kinda problematic since it became unavoidable to run KDE.

What happens in such cases:

– Activities don’t work (the bar is empty),
– You can’t add widgets (same behavior),

I suppose installing kde-workspace in a different prefix than kdelibs is enough to reproduce this issue.

There are probably lots of other minor or real issues that make packagers cry every night, add them in the comments, maybe I’ll write a part#2🙂

Written by krop

6 November 2012 at 23:22

Posted in KDE

12 Responses

Subscribe to comments with RSS.

  1. Amen!

    Yet another packager who suffers the same problems.

  2. Regarding relying on the Oxygen icons I have to continue to disagree, at least for pragmatic reasons:
    The problem is that you never know what icons a library uses internally, at least for the paths in the code your program might possibly trigger. Especially as the library might be updated to another version, with more pimped dialogs (so using new icons).
    So not only programs, but also libraries would need to install into the hi-color theme. But for the kdelibs this is not done, these libraries de-facto rely on the Oxygen icons.
    So if kdelibs relies on them, my program can as well.
    In theory you are right, I agree. But reality rules.

    Friedrich

    7 November 2012 at 01:25

    • This discussion is mixing up 2 unrelated issues: Relying on Oxygen being installed and installing the application’s .desktop file icon to oxygen rather than hicolor. The first is acceptable, the second is not.

      Kevin Kofler

      7 November 2012 at 01:51

  3. What you wrote makes a lot of sense. However, for your point 1, IMHO, the real issue is that there are runtime modules in the first place, as opposed to the library packages also shipping what they need at runtime. That by itself is already incompatible with how distros work: Normally, when package P needs library L, we use in P a BuildRequires: L-devel, and RPM (in our case) will automatically detect a dependency on L. But now if L doesn’t actually work without runtime package R, we have only 2 options:
    a. Add a Requires: R to L. But this really sucks because R typically needs L-devel to build and L to run, and thus it creates a circular dependency with the usual bootstrapping nightmares.
    b. Do not add a Requires: R to L. Instead, expect all the packages using L to manually add Requires: R. This is what we currently do for kde-runtime in Fedora. This sucks because the Requires invariably gets forgotten in many packages.
    Examples of this problem are L=kdelibs, R=kde-runtime and L=kdelibs, R=kate-part. It would make things a lot easier for us distributions if libraries actually carried their runtime requirements in the SAME tarball so that we can ship them in the same package and avoid the circular dependency. (In the kate-part case, this was actually changed for the worse.)

    I’ll also point out that at least for us in Fedora, point 6 doesn’t really matter. We install everything to the same prefix anyway. But that doesn’t mean custom prefixes shouldn’t work for those users who do rely on them.

    Kevin Kofler

    7 November 2012 at 02:03

  4. “But what would happen if you try to run KDE with such a setup”

    KDE is the community.

    “This is kinda problematic since it became unavoidable to run KDE.”

    If you are using “KDE” to mean “the desktop KDE makes” then yes, exiomatically Plasma is core to Plasma.

    “- Activities don’t work (the bar is empty),”

    while the switcher UI was not there, activities were probably working just fine.

    in any case, Package in libplasma2 does use standard dirs searching. i’m not going to change how it works in libplasma1 as the setup you describe is indeed non-standard and the chance for breakages is too high.

    Aaron Seigo

    7 November 2012 at 02:18

  5. […] Things you should double check twice (or more) before releasing your software When you start packaging more and more things for your distribution, you realize how things would be easier if upstream was slighly more concerned about how distributions work. […] […]

  6. You can find more things to make distros happy here and in the links section of it:

    http://wiki.debian.org/UpstreamGuide

    Anonymous

    7 November 2012 at 04:44

  7. Turning this into a “why”, so we can be productive, I can explain why these mistakes happen.

    From my own personal experience I went from some guy doing a bit of C++ hacking away happily to suddenly I’m meant to know all sorts of packaging, icon naming conventions, how translations work, what to install where, how to structure tarballs. There is _so much_ to do in a release that is just taken for granted.

    There’s no training, no-one tells you and documentation is scattered and sparse. The only time I know I’ve done anything wrong is when people shout at me…which is way too late.

    One thing that has been great for me is the Gentoo packager johu (sorry, don’t know his real name) who has always been really patient and helped me test packages as soon as I release them before I make announcements. The community are really helpful when asked.

    Blog posts like this are great , if viewed not as a rant but as a checklist to go through and lessons to learn from. Make sure you update techbase to help the people replacing me making new things have things to read. Within a week this will be at the bottom of the PlanetKDE and never read again. Which goes back to the no documentation for the developers of tomorrow.

    Also it would be great if packagers try and package beta releases of KDE, that way we can catch things before it’s too late.

    David Edmundson

    7 November 2012 at 06:01

    • +1

      annma

      8 November 2012 at 09:13

  8. libpcre16 not the same is libpcre. Qt needs libpcre16, you selected -system-pcre yet you don’t have libpcre16, so the build fails.

    That is entirely your fault. Maybe the fault with qt is all yours?

    Oh well another clueless packager.

    Anonymous

    7 November 2012 at 14:01

  9. Could you imagine that there might be reasons that _patched_ versions of 3rd party libs are included?

    You also correctly noticed that in Botan case the library is essentially a single file. I trust people are really looking forward to provide options for excluding every single file in the project in case some packager might consider replacing it.

    Anonymous

    7 November 2012 at 20:17

    • Could you imagine that there are reasons why we prefer using our packages rather than embedded 3rd party libraries ?

      If a bug lies in Botan, submit the bugfix to Botan, we’ll gladly pick it up if it’s accepted and is useful. We’ll then patch *our* botan package so that the fix is available for everyone.

      krop

      7 November 2012 at 21:16


Comments are closed.

%d bloggers like this: