Thursday, April 16, 2009

similar install (part 2)

Following on from the earlier posting the script is pretty much complete and ran okay giving me 416 packages to install to bring the mini9 in line with the Inspiron 1525 machine.

## Given the output of dpkg --get-selections this program filters
## out those entries that are not installable.
## In order to have a program that is less dependent on particular version of
## python-apt, this script sometimes avoids making use of python-apt Cache() class
## inbuilt iteration functionality.
## (c) 2009 Gary Wright
##This program is free software: you can redistribute it and/or modify
##it under the terms of the GNU General Public License as published by
##the Free Software Foundation, either version 3 of the License, or
##(at your option) any later version.
##This program is distributed in the hope that it will be useful,
##but WITHOUT ANY WARRANTY; without even the implied warranty of
##GNU General Public License for more details.
##You should have received a copy of the GNU General Public License
##along with this program. If not, see
## V1.0 20090515GW
import apt
import sys, fileinput
import logging as l # avoid reserved mathematical word 'log'

# Log to stderr - leaving the output uncluttered and suitable for input to dpkg.
debug = 3 # 2 rather than 1 gives you more chat. 3 for testing. 0 for silence.

all_packages = apt.Cache()
installed_packages = [i for i in all_packages if i.isInstalled]
installed_packages_list = []
for i in installed_packages:
upgradable_packages = [i for i in installed_packages if i.isUpgradable]
if debug: l.debug("Total available packages:" + str(len(all_packages)) )
if debug: l.debug("Total installed packages:" + str(len(installed_packages)) )

def output_selections():
"""Output using print in a format that is suitable for giving to dpkg"""
for i in install_list:
print "%s\t\t\t\t\t\tinstall" % i

def list_intersection(list1,list2):
"""Generator function which for every item in list 1 checks that the
item exists also in list 2.
for item1 in list1:
if item1 in list2: yield item1

install_list = []
for line in fileinput.input():
line_split = line.strip().split('\t') # last word on the line is the action
pkgname = line_split[0]
action = line_split[-1]
if action == 'install':
if debug > 3: l.debug(action)
given_list=[ '' ]
all_packages_list = all_packages.keys()

if debug: l.debug("given_list:" + str(len(given_list)) )
if debug: l.debug("install_list:" + str(len(install_list)) )
# li short for list item
install_list = [ li for li in \
list_intersection(install_list[:],all_packages.keys()) ]
if debug: l.debug("install_list:" + str(len(install_list)))
if (debug > 3): l.debug("install_list:\n" + "\n".join(install_list))

if (debug > 3): # debug > 3 is testing only so safely skipread this portion
for i in upgradable_packages:
print (, i.candidateVersion, i.architecture, i.installedPriority)

if (debug > 2): # debug > 2 is unlikely to be in production so safely skipread this portion
install_list = list(set(install_list).intersection(all_packages_list))
l.debug("install_list using casting to and from sets:" + str(len(install_list)))
install_list_new = [i for i in all_packages if not i.isInstalled]
l.debug("install_list_new:" + str(len(install_list_new)) )
install_list_new = [ li for li in \
list_intersection(install_list[:],install_list_new[:]) ]
l.debug("install_list_new:" + str(len(install_list_new)) )
install_list_new_setty = list(set(install_list) - set(installed_packages_list))
l.debug("install_list_new_setty:" + str(len(install_list_new_setty)) )
count = 0
for i in install_list_new_setty:
if (count == 0): lineout = "apt-get --no-act install "
lineout += i.ljust(len(i)+1)
count += 1
if (count == 30):
count = 0
if (lineout.strip().split()[-1] != "install"):l.debug(lineout)
if (debug > 3): l.debug("evince" in installed_packages_list)
if (debug > 3): l.debug("evince" in install_list_new_setty)

Test execution of the script:
echo -e "bash\t\t\t\t\t\tinstall" | python -W ignore ./ 2>~/python.stderr
echo -e "rash\t\t\t\t\t\tinstall" |
python -W ignore ./ 2>~/python.stderr

Further reference or alternative ways of list intersection:
# shows how to use filter to do
# intersection = filter(lambda x:x in list1, list2)

# msg34179 gives notes about pythons inbuilt set object and methods

Reference documentation for fileinput helped me with the file reading.

For execution on the mini9 I piped in the output from dpkg --get-selections and this gave the 416 line output telling me what new installs I might make.

That output can be written to a file and piped back into dpkg --set-selections but I have not tried that just yet.

What I have done is manually examine ~/python.stderr and the apt-get statements generated when debug is set to 3 and pruned down that list reviewing as I go. That way the mini9 avoids a bit of the cruft or machine specific packages that might not be desirable on the mini9.

Change debug to 2 or 1 in the python code will cut down on the messages written to ~/python.stderr (or wherever you might redirect stderr to).

Note: The mail address given in the code is yet to be operational and will be usable when the server setup is complete in 2010.

No comments: