ubuntu-appstore-developers team mailing list archive

[Sebastian Heinlein <glatzor@xxxxxxxxxx>]

Am Mittwoch, den 03.07.2013, 14:11 +0100 schrieb Colin Watson:

> > AFAIK the required API of the AppStore would be quite small and the
> > package downloading and update calculation is done in the GUI component,
> > right? So we could end up with a very small set of methods e.g.
> > InstallFile() RemovePackages(), GetPackages(FILTER_INSTALLED)?
> 
> I believe so, at least as a first pass.  These would roughly map to
> "click install", "click remove", and (forthcoming) "click list".

The native Aptdaemon D-Bus interface doesn't support querying the
package cache. It was designed for a sophisticated client application
that accesses the cache directly and only makes use of the daemon for
the higher privileged manipulations of the cache.

So if you want to have a click list method I would suggest using the
PackageKit interface. Independently of the question if the interface is
provided by aptdaemon or packagekit.

I just wrote a small click backend for PackageKit as a proof of concept.
You can find the patch attached to this mail.

> There are some complexities relating to per-user installation of
> packages, which is a requirement: that is, all packages will be unpacked
> as a privileged user, but it's possible to unpack multiple parallel
> versions of a package and each user will be able to register their own
> "current version" links.  Do you have any thoughts on whether this kind
> of dual operation (unpack as privileged user, link as ordinary user)
> should be done as two D-Bus methods or one?

> I was indeed looking at aptdaemon for this.  As an alternative to
> replacing the whole worker, do you think it might make sense to modify
> apt.debfile.DebPackage.install to understand click packages (a rather
> small change, at least if it only handles the privileged side of
> things)?  Since they use the .deb container format, that seems like a
> convenient class to reuse.  Part of the point of using the same
> container format was to reduce the amount of work we needed to do at
> this kind of layer, for code that's fundamentally not that interesting.

Replacing the "whole" worker isn't very intrusive. That is already done
by the PackageKit compat mode :)

Why would you need the debfile.DebPackage class at all? Since you don't
have got any dependency information you could call the click command
directly, right? The DebPackage is used to calculate the dependencies
that need to be installed from the archive. The DebPackage class would
only be interesting if the click-package command uses a status fd as
dpkg/apt do.

So after the 1.1 release of aptdaemon. I could refactor the workers a
little bit to remove the dependency of python-apt.

> > Furthermore it limits the click package implementation to Python or
> > requires a Python library to access the click packages system.
> 
> Not really.  It would make sense to invoke /usr/bin/click as a
> subprocess, which the relevant parts of aptdaemon can already do without
> much difficulty, and as mentioned above we can reuse existing Python
> libraries for things like handling some metadata.  But it depends how
> much sophistication you think aptdaemon would need.

How much sophistication does the design team need? E.g. progress and
status reporting? How to handle errors of the click command?

> > On the other hand there is the PackageKit daemon and its D-Bus API. The
> > PackageKit daemon allows to write backends in different languages: the
> > current Python based click package system could use a spawned backend.
> > If it is still the plan, click packages could be re-written in C/C++
> > later without having to do any changes on the communication between the
> > daemon and the client.
> 
> I'm not sure I understand how this isn't already the case with
> aptdaemon.  Can you elaborate?

If you re-write a PackageKit backend the daemon-client-communication
will be the same. You don't have to re-implement the whole communication
in another language - only the backend.

> > The support for the latest PackageKit 0.8 system D-Bus interface landed
> > in aptdaemon's trunk today. I plan to release and upload a 1.1 version
> > of aptdaemon soon. This will unblock the transition to packagekit 0.8.9
> > for saucy.
> 
> That would be great, and let me know if I can help with that.  I'm
> really agnostic as to whether we use aptdaemon or PackageKit as the
> backend; I think it definitely makes sense to use one of them rather
> than inventing some new service, but I wanted to avoid getting caught up
> in the PackageKit transition [...]

I fixed some bugs recently and the 1.1 of aptdaemon with the PK 0.8 API
gets into shape.

> > PackageKit provides two D-Bus interfaces to manage software. The system
> > D-Bus interface should be used by the native client libraries. For
> > applications that cannot access the client libraries or don't want to
> > provide a separate graphical user interface that handles errors or shows
> > the progress there is also a small session D-Bus interface which is
> > implemented by gnome-packagekit and apper.
> 
> Do we need to use a session interface for the kinds of per-user work I
> described?

You could mis/re-use the InstallPackages transaction type to set the
link to the version of the package that the user wants to use.

Cheers,

Sebastian

>From 20fe23b2cd459b51647b13d43bdd3d0a3a9ee2ba Mon Sep 17 00:00:00 2001
From: Sebastian Heinlein <devel@xxxxxxxxxx>
Date: Sat, 6 Jul 2013 22:18:58 +0200
Subject: [PATCH] click: Add a very basic click package backend

---
 backends/Makefile.am              |    4 +
 backends/click/Makefile.am        |   21 ++++
 backends/click/clickBackend.py    |   67 +++++++++++
 backends/click/pk-backend-click.c |  221 +++++++++++++++++++++++++++++++++++++
 configure.ac                      |   12 +-
 5 files changed, 324 insertions(+), 1 deletion(-)
 create mode 100644 backends/click/Makefile.am
 create mode 100755 backends/click/clickBackend.py
 create mode 100644 backends/click/pk-backend-click.c

diff --git a/backends/Makefile.am b/backends/Makefile.am
index 68579c9..b5a3f8a 100644
--- a/backends/Makefile.am
+++ b/backends/Makefile.am
@@ -16,6 +16,10 @@ if BACKEND_TYPE_BOX
 SUBDIRS += box
 endif
 
+if BACKEND_TYPE_CLICK
+SUBDIRS += click
+endif
+
 if BACKEND_TYPE_CONARY
 SUBDIRS += conary
 endif
diff --git a/backends/click/Makefile.am b/backends/click/Makefile.am
new file mode 100644
index 0000000..df5b2a0
--- /dev/null
+++ b/backends/click/Makefile.am
@@ -0,0 +1,21 @@
+helperdir = $(datadir)/PackageKit/helpers/click
+dist_helper_DATA = 			\
+	clickBackend.py			\
+	$(NULL)
+
+plugindir = $(PK_PLUGIN_DIR)
+plugin_LTLIBRARIES = libpk_backend_click.la
+libpk_backend_click_la_SOURCES = pk-backend-click.c
+libpk_backend_click_la_LIBADD = $(PK_PLUGIN_LIBS)
+libpk_backend_click_la_LDFLAGS = -module -avoid-version
+libpk_backend_click_la_CFLAGS = $(PK_PLUGIN_CFLAGS) $(WARNINGFLAGS_C)
+
+install-data-hook:
+	chmod a+rx $(DESTDIR)$(helperdir)/*.py
+
+clean-local:
+	rm -f *.pyc
+	rm -f *~
+
+
+-include $(top_srcdir)/git.mk
diff --git a/backends/click/clickBackend.py b/backends/click/clickBackend.py
new file mode 100755
index 0000000..4a0593b
--- /dev/null
+++ b/backends/click/clickBackend.py
@@ -0,0 +1,67 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+"""Provides an apt backend to PackageKit
+
+Copyright (C) 2013 Sebastian Heinlein <glatzor@xxxxxxxxxx>
+
+Licensed under the GNU General Public License Version 2
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+"""
+
+__author__ = "Sebastian Heinlein <devel@xxxxxxxxxx>"
+
+
+import sys
+
+from clickpackage.install import ClickInstaller
+from packagekit import backend, enums
+
+
+CLICK_ROOT = "/opt/click.ubuntu.com"
+
+
+class ClickPackageKitBackend(backend.PackageKitBaseBackend):
+
+    """This class provides a PackageKit backend for the click
+    packages introduced in the Ubuntu Phone.
+    """
+
+    def install_files(self, transaction_flags, install_files):
+        """Install downloaded package files."""
+        self.percentage(None)
+        self.status(enums.STATUS_INSTALL)
+        self.allow_cancel(False)
+
+        installer = ClickInstaller(root=CLICK_ROOT,
+                                   force_missing_framework=False)
+
+        for file_path in install_files:
+            try:
+                pkg_name, pkg_version = installer.audit(file_path)
+            except Exception as error:
+                self.error(enums.ERROR_INVALID_PACKAGE_FILE, str(error))
+                break
+            self.package("%s;%s;;" % (pkg_name, pkg_version),
+                         enums.INFO_INSTALLING,
+                         "")
+            try:
+                installer.install(file_path)
+            except Exception as error:
+                self.error(enums.ERROR_INTERNAL_ERROR, str(error))
+                break
+            self.package("%s;%s;;" % (pkg_name, pkg_version),
+                         enums.INFO_INSTALLED,
+                         "")
+
+def main():
+   backend = ClickPackageKitBackend("")
+   backend.dispatcher(sys.argv[1:])
+
+if __name__ == "__main__":
+    main()
+
+# vim: es=4 et sts=4
diff --git a/backends/click/pk-backend-click.c b/backends/click/pk-backend-click.c
new file mode 100644
index 0000000..63aa080
--- /dev/null
+++ b/backends/click/pk-backend-click.c
@@ -0,0 +1,221 @@
+/* -*- Mode: C; tab-width: 8; indent-tabs-mode: t; c-basic-offset: 8 -*-
+ *
+ * Copyright (C) 2007-2010 Richard Hughes <richard@xxxxxxxxxxx>
+ * Copyright (C) 2013 Sebastian Heinlein <devel@xxxxxxxxxx>
+ *
+ * Licensed under the GNU General Public License Version 2
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+
+#include <config.h>
+#include <pk-backend.h>
+#include <pk-backend-spawn.h>
+#include <string.h>
+
+typedef struct {
+	PkBackendSpawn	*spawn;
+} PkBackendClickPrivate;
+
+static PkBackendClickPrivate *priv;
+
+/**
+ * pk_backend_get_description:
+ */
+const gchar *
+pk_backend_get_description (PkBackend *backend)
+{
+	return "Click";
+}
+
+/**
+ * pk_backend_get_author:
+ */
+const gchar *
+pk_backend_get_author (PkBackend *backend)
+{
+	return "Sebastian Heinlein <devel@xxxxxxxxxx>";
+}
+
+/**
+ * pk_backend_stderr_cb:
+ */
+static gboolean
+pk_backend_stderr_cb (PkBackendJob *job, const gchar *output)
+{
+	if (strstr (output, "DeprecationWarning") != NULL)
+		return FALSE;
+	return TRUE;
+}
+
+/**
+ * pk_backend_stdout_cb:
+ */
+static gboolean
+pk_backend_stdout_cb (PkBackendJob *job, const gchar *output)
+{
+	return TRUE;
+}
+
+
+/**
+ * pk_backend_initialize:
+ * This should only be run once per backend load, i.e. not every transaction
+ */
+void
+pk_backend_initialize (PkBackend *backend)
+{
+	/* create private area */
+	priv = g_new0 (PkBackendClickPrivate, 1);
+
+	g_debug ("backend: initialize");
+	priv->spawn = pk_backend_spawn_new ();
+	pk_backend_spawn_set_filter_stderr (priv->spawn, pk_backend_stderr_cb);
+	pk_backend_spawn_set_filter_stdout (priv->spawn, pk_backend_stdout_cb);
+	pk_backend_spawn_set_name (priv->spawn, "click");
+	pk_backend_spawn_set_allow_sigkill (priv->spawn, FALSE);
+}
+
+/**
+ * pk_backend_destroy:
+ * This should only be run once per backend load, i.e. not every transaction
+ */
+void
+pk_backend_destroy (PkBackend *backend)
+{
+	g_debug ("backend: destroy");
+	g_object_unref (priv->spawn);
+	g_free (priv);
+}
+
+/**
+ * pk_backend_start_job:
+ */
+void
+pk_backend_start_job (PkBackend *backend, PkBackendJob *job)
+{
+	if (pk_backend_spawn_is_busy (priv->spawn)) {
+		pk_backend_job_error_code (job,
+					   PK_ERROR_ENUM_LOCK_REQUIRED,
+					   "spawned backend requires lock");
+		pk_backend_job_finished (job);
+		return;
+	}
+}
+
+/**
+ * pk_backend_get_filters:
+ */
+PkBitfield
+pk_backend_get_filters (PkBackend *backend)
+{
+	return pk_bitfield_from_enums (
+		PK_FILTER_ENUM_INSTALLED,
+		-1);
+}
+
+/**
+ * pk_backend_get_roles:
+ */
+PkBitfield
+pk_backend_get_roles (PkBackend *backend)
+{
+	return pk_bitfield_from_enums (
+		PK_ROLE_ENUM_GET_PACKAGES,
+		PK_ROLE_ENUM_INSTALL_FILES,
+		PK_ROLE_ENUM_REMOVE_PACKAGES,
+		-1);
+}
+
+/**
+ * pk_backend_get_mime_types:
+ */
+gchar **
+pk_backend_get_mime_types (PkBackend *backend)
+{
+	const gchar *mime_types[] = {
+				"application/x-deb",
+				NULL };
+	return g_strdupv ((gchar **) mime_types);
+}
+
+/**
+ *  * pk_backend_get_groups:
+ *   */
+PkBitfield
+pk_backend_get_groups (PkBackend *backend)
+{
+	return pk_bitfield_from_enums (
+		PK_GROUP_ENUM_UNKNOWN,
+		-1);
+}
+
+/**
+ * pk_backend_get_packages:
+ */
+void
+pk_backend_get_packages (PkBackend *backend, PkBackendJob *job, PkBitfield filters)
+{
+	gchar *filters_text;
+	filters_text = pk_filter_bitfield_to_string (filters);
+	pk_backend_spawn_helper (priv->spawn, job, "clickBackend.py", "get-packages", filters_text, NULL);
+	g_free (filters_text);
+}
+
+/**
+ * pk_backend_install_files:
+ */
+void
+pk_backend_install_files (PkBackend *backend, PkBackendJob *job, PkBitfield transaction_flags, gchar **full_paths)
+{
+	gchar *package_ids_temp;
+	gchar *transaction_flags_temp;
+	package_ids_temp = g_strjoinv (PK_BACKEND_SPAWN_FILENAME_DELIM, full_paths);
+	transaction_flags_temp = pk_transaction_flag_bitfield_to_string (transaction_flags);
+	pk_backend_spawn_helper (priv->spawn, job,
+				 "clickBackend.py",
+				 "install-files",
+				 transaction_flags_temp,
+				 package_ids_temp,
+				 NULL);
+	g_free (package_ids_temp);
+	g_free (transaction_flags_temp);
+}
+
+/**
+ * pk_backend_remove_packages:
+ */
+void
+pk_backend_remove_packages (PkBackend *backend, PkBackendJob *job,
+			    PkBitfield transaction_flags,
+			    gchar **package_ids,
+			    gboolean allow_deps,
+			    gboolean autoremove)
+{
+	gchar *package_ids_temp;
+	gchar *transaction_flags_temp;
+	package_ids_temp = pk_package_ids_to_string (package_ids);
+	transaction_flags_temp = pk_transaction_flag_bitfield_to_string (transaction_flags);
+	pk_backend_spawn_helper (priv->spawn, job,
+				 "clickBackend.py",
+				 "remove-packages",
+				 transaction_flags_temp,
+				 package_ids_temp,
+				 pk_backend_bool_to_string (allow_deps),
+				 pk_backend_bool_to_string (autoremove),
+				 NULL);
+	g_free (package_ids_temp);
+	g_free (transaction_flags_temp);
+}
diff --git a/configure.ac b/configure.ac
index b26bd05..12c34da 100644
--- a/configure.ac
+++ b/configure.ac
@@ -490,6 +490,7 @@ AC_ARG_ENABLE(alpm, AS_HELP_STRING([--enable-alpm],[use the ALPM backend]),enabl
 AC_ARG_ENABLE(apt, AS_HELP_STRING([--enable-apt],[use the APT backend]),enable_apt=$enableval,enable_apt=no)
 AC_ARG_ENABLE(aptcc, AS_HELP_STRING([--enable-aptcc],[use the APTcc backend]),enable_aptcc=$enableval,enable_aptcc=no)
 AC_ARG_ENABLE(box, AS_HELP_STRING([--enable-box],[use the BOX backend]),enable_box=$enableval,enable_box=no)
+AC_ARG_ENABLE(click, AS_HELP_STRING([--enable-click],[use the Click backend]),enable_click=$enableval,enable_click=no)
 AC_ARG_ENABLE(conary, AS_HELP_STRING([--enable-conary],[use the CONARY backend]),enable_conary=$enableval,enable_conary=no)
 AC_ARG_ENABLE(dummy, AS_HELP_STRING([--enable-dummy],[use the dummy backend]),enable_dummy=$enableval,enable_dummy=yes)
 AC_ARG_ENABLE(entropy, AS_HELP_STRING([--enable-entropy],[use the entropy backend]),enable_entropy=$enableval,enable_entropy=no)
@@ -511,6 +512,7 @@ AM_CONDITIONAL(BACKEND_TYPE_ALPM, [test x$enable_alpm = xyes])
 AM_CONDITIONAL(BACKEND_TYPE_APT, [test x$enable_apt = xyes])
 AM_CONDITIONAL(BACKEND_TYPE_APTCC, [test x$enable_aptcc = xyes])
 AM_CONDITIONAL(BACKEND_TYPE_BOX, [test x$enable_box = xyes])
+AM_CONDITIONAL(BACKEND_TYPE_CLICK, [test x$enable_click = xyes])
 AM_CONDITIONAL(BACKEND_TYPE_CONARY, [test x$enable_conary = xyes])
 AM_CONDITIONAL(BACKEND_TYPE_DUMMY, [test x$enable_dummy = xyes])
 AM_CONDITIONAL(BACKEND_TYPE_ENTROPY, [test x$enable_entropy = xyes])
@@ -580,7 +582,7 @@ dnl ---------------------------------------------------------------------------
 AC_ARG_WITH([default_backend],
 	    AS_HELP_STRING([--with-default-backend=<option>],
 			   [Default backend to use
-                           auto,alpm,apt,aptcc,box,conary,dummy,entropy,hawkey,opkg,pisi,portage,ports,slapt,smart,urpmi,yum,zif,zypp (dummy)]))
+                           auto,alpm,apt,aptcc,box,click,conary,dummy,entropy,hawkey,opkg,pisi,portage,ports,slapt,smart,urpmi,yum,zif,zypp (dummy)]))
 # default to a sane option for the installed tool
 if test x$with_default_backend = x; then
 	if test -f /usr/bin/yum ; then
@@ -589,6 +591,8 @@ if test x$with_default_backend = x; then
 		with_default_backend=zif
 	elif test -f /usr/lib/libalpm.so; then
 		with_default_backend=alpm
+	elif test -f /usr/bin/click-package ; then
+		with_default_backend=click
 	elif test -f /usr/bin/apt-get ; then
 		with_default_backend=aptcc
 	elif test -f /usr/bin/conary ; then
@@ -757,6 +761,10 @@ if test x$enable_slapt = xyes; then
 	AC_SUBST(SLAPT_LIBS)
 fi
 
+if test x$enable_apt = xyes; then
+	PY_CHECK_MOD([clickpackage],,,AC_MSG_ERROR([Click backend needs python3-clickpackage.install]))
+fi
+
 AC_SUBST(PK_CONF_DIR, "\$(sysconfdir)/PackageKit")
 AC_SUBST(PK_YUM_PLUGIN_DIR, "\$(prefix)/lib/yum-plugins")
 AC_SUBST(PK_DB_DIR, "\$(localstatedir)/lib/PackageKit")
@@ -792,6 +800,7 @@ backends/alpm/Makefile
 backends/apt/Makefile
 backends/aptcc/Makefile
 backends/box/Makefile
+backends/click/Makefile
 backends/conary/Makefile
 backends/dummy/Makefile
 backends/entropy/Makefile
@@ -872,6 +881,7 @@ echo "
         APTcc backend:             ${enable_aptcc}
         BOX backend:               ${enable_box}
         CONARY backend:            ${enable_conary}
+        Click backend:             ${enable_click}
         dummy backend:             ${enable_dummy}
         Entropy backend:           ${enable_entropy}
         Hawkey backend:            ${enable_hawkey}
-- 
1.7.10.4