Ubuntu: How to find manually installed packages?



Question:

I installed a plain Ubuntu 10.10 on my computer and installed some software via apt-get. Last week I managed to break everything and just started from scratch, and I need to reinstall my software. Is there some way to create a list with all the packages that I have installed manually?

So that it would give me a list like texlive, ... and not texlive, texlive-dep1, textlive-dep2, ... and all the standard packages removed?

If I could somehow figure out which programs out of the regular install I have removed, that would be awesome too!


Update 2015-05-23: I now use Ansible as configuration management on my systems. There I specify the packages to install. This serves the goal, installing all the programs on a fresh system, even better. So this question might be the entry into some light configuration management.


Solution:1

With this suggestion, I'm assuming your old installation will still boot!

To replicate one set of packages on another machine:

On System A, run:

dpkg --get-selections | grep -v deinstall > my-selections  

Move the my-selections file over to System B.

On System B, run:

dpkg --set-selections < my-selections  

and then:

sudo apt-get dselect-upgrade  

Important note: if you have installed packages from non-standard repositories and/or PPAs, you will also want to copy /etc/apt/sources.list and the contents of /etc/apt/sources.list.d/ from System A to System B before you run the upgrade.

You can use dpkg to see what you've removed as well (NB: this will also include packages that you manually installed and removed):

dpkg --get-selections | grep deinstall  

You can see your results in the terminal, or, of course, redirect to a file.


Solution:2

This thread from superuser.com gives this solution:

aptitude search '?installed ?not(?automatic)'  


Solution:3

If your apt logs are in /var/log/apt, something like this should work:

gunzip -c /var/log/apt/history.log.*.gz | grep "apt-get install"  

Or if you want to get rid of some of the repetitive junk in the output:

gunzip -c /var/log/apt/history.log.*.gz | grep "apt-get install" \    | cut -f4- -d" " | sort | uniq  


Solution:4

You could use apt-mark, but I recommend debfoster:

sudo apt-get install debfoster  sudo debfoster  

This will inspect all installed packages and figure out which ones are keeping the others installed:

texlive-full is keeping the following 161 packages installed:    cm-super cm-super-minimal context doc-base dvipng feynmf    fonts-gfs-artemisia fonts-gfs-baskerville fonts-gfs-bodoni-classic    ...  Keep texlive-full? [Ynpsiuqx?], [H]elp:  

As you answer "y" to each question (just push Enter to move quickly), debfoster will collect the package list and write them line-by-line to a file. By default this is at /var/lib/debfoster/keepers. It looks like this:

gnome-do  texlive-full  ...  

I configure debfoster via /etc/debfoster.conf to put this list at /etc/debfoster-keepers and track the file with etckeeper to keep history and backups. The answer here shows how to install a list of packages from a newline-delimited text file:

sudo apt-mark manual $(cat debfoster-keepers)  

Note a limitation here, packages you purged have a '-' in front of them. So you want to remove those lines before calling apt-mark.

Even though the debfoster's website says that debfoster is deprecated in favor of aptitude, I prefer debfoster's prompt and simple configuration. It puts you in the middle of your package database and lets you clean things up, making the auto and manual packages more obvious.

Type "h" at the debfoster prompt to explain your options. Type "?" to see the package description. The how-to here might be useful.


Solution:5

comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)  

Gives all manually installed packages (not system packages, not dependencies). For examples it shows build-essential but not gcc.


Solution:6

I finally got it now:

outfile="$(mktemp)"  pattern='(\[INSTALLIEREN\]|\[INSTALL\])'    if [[ -f "/var/log/aptitude.1.gz" ]]  then          gunzip -c /var/log/aptitude.*.gz | grep -E "$pattern" | awk '{ print $2; }' > "$outfile"  fi    if [[ -f "/var/log/aptitude" ]]  then          grep -E "$pattern" "/var/log/aptitude" | awk '{ print $2; }' >> "$outfile"  fi    sort "$outfile"  rm "$outfile"  


Solution:7

I'm able to pull everything by opening the log files in /var/log/apt/

I then go in and manually filter out the apt-get install packages. There may be a way to do this programatically but I'm not aware of it.


Solution:8

I didn't see any solutions here work for me, I have installed quite a few deb packages with dpkg and a few of the items I was particularly looking for were missing.

A rather lengthy one liner, but convenient to copy and paste would be:

export DPKG_INITIAL=$(mktemp) DPKG_INSTALLED=$(mktemp) DPKG_CUSTOM=$(mktemp) DPKG_DEPS=$(mktemp) zgrep -E '^Package' /var/log/installer/initial-status.gz | awk '{ print $2 }' > $DPKG_INITIAL ; awk '$3 !~ /install|remove|purge/ { next } { gsub(/remove|purge/, "uninstall", $3) ; gsub(/:.+/, "", $4) ; a[$4]=$3 } END { for (p in a) { if (a[p] == "install") { print p } } }' /var/log/dpkg.log | sort -u > $DPKG_INSTALLED ; comm -23 installed initial > $DPKG_CUSTOM ; function rdep() { apt-cache rdepends $1 | tail -n +3 | sed -e 's/^ //' -e '/^ /d' | cut -d':' -f1 | sort -u; } ; echo "$(for i in $(cat custom) ; do rdep $i ; done)" | sort -u > $DPKG_DEPS ; comm -23 custom deps > my-packages ; rm $DPKG_INITIAL $DPKG_INSTALLED $DPKG_CUSTOM $DPKG_DEPS  

The above command saves a list of packages to a file in your current working directory named my-packages.

Explanation

I first built a list of packages that composed the baseline of packages selected during installation.

zgrep -E '^Package' /var/log/installer/initial-status.gz | awk '{ print $2 }' > initial  

Followed by a long list of items installed in general.

awk '$3 !~ /install|remove|purge/ { next } { gsub(/remove|purge/, "uninstall", $3) ; gsub(/:.+/, "", $4) ; a[$4]=$3 } END { for (p in a) { if (a[p] == "install") { print p } } }' /var/log/dpkg.log | sort -u > installed  

I then compared the two files initial and installed to list only the items unique to installed.

comm -23 installed initial > custom  

From there I wanted to filter out the dependencies, this is where this method may miss some desired packages, it is unaware of dependencies that are also explicitly installed.

I wrote a quick bash function to shorten this steps in processing these items.

function rdep() { apt-cache rdepends $1 | tail -n +3 | sed -e 's/^ //' -e '/^ /d' | cut -d':' -f1 | sort -u; }  

After that I passed each line from my file custom into this function with xargs.

echo "$(for i in $(cat custom) ; do rdep $i ; done)" | sort -u > deps  

Once I had the long list of every possible dependency, (not sure on the every possible statement), I once again got the lines that were unique to a single file.

comm -23 custom deps > manual  

And my finished list of packages is now in a file named manual available for me to review.


Solution:9

For those who prefer using apt-* tools, there's an apt-mark utility that allows both querying and modifying manual/auto state.

apt-mark showmanual  

You can also specify pattern for packages you interested in:

apt-mark showmanual qemu\*  


Solution:10

cd /var/log/apt  cat  history.log | grep Commandline  

this way you see the list of commands performed in the past.

Remove the grep code if you need more information:

less history.log  

if the log is compressed (i.e. end with gz)

gunzip <filename>  

to zip it again when finished you can do:

gzip <filename>  


Solution:11

Someone linked to https://unix.stackexchange.com/questions/3595/ubuntu-list-explicitly-installed-packages/3624#3624 which does have a good solution, but, it doesn’t behave correctly as the output from aptitude has changed. Here is an updated version, based around compared currently installed packages compared to 12.04 LTS. You will need aptitude installed, that is the only requirement.

aptitude search '~i !~M' -F '%p' | sort -u | tr -d ' ' > currentlyinstalled && wget -qO - http://mirror.pnl.gov/releases/precise/ubuntu-12.04.3-desktop-amd64.manifest | cut -f 1 | sort -u > defaultinstalled && comm -23 currentlyinstalled defaultinstalled  

To break the above command down into parts, this bit outputs one package per line of everything installed on the system

aptitude search '~i !~M' -F '%p' | sort -u | tr -d ' ' > currentlyinstalled  

And this downloads the default package list & crops the redundant information.

wget -qO - http://mirror.pnl.gov/releases/precise/ubuntu-12.04.3-desktop-amd64.manifest | cut -f 1 | sort -u > defaultinstalled  

and comm compares the two files and outputs the packages which arent in the default list.

comm -23 currentlyinstalled defaultinstalled  


Solution:12

Use a distribution manifest file as the base package set. Sort the manually installed results into package groups by architecture and section, so it's easier to focus on groups of packages (maybe you don't care about some sections).

https://gist.github.com/darrenleeweber/8cc570ff402f19af7fa4

#!/bin/bash    manifest_url='http://releases.ubuntu.com/releases/trusty/ubuntu-14.04.3-desktop-amd64.manifest'  manifest_file=$(echo $manifest_url | sed -e 's#.*/##g')  if [ ! -e $manifest_file ]; then      wget -q $manifest_url  fi  cat $manifest_file | cut -f1 | sort -u > default_installed.txt    aptitude search '~i !~M' -F '%p' --disable-columns | sort -u > currently_installed.txt    comm -23 currently_installed.txt default_installed.txt > manually_installed.txt    # sort the 'mannually_installed.txt' packages by architecture and section  mkdir -p package_files  while read p; do      apt-cache show $p > info.txt      arch=$(grep -m1 'Architecture: ' info.txt | sed -e 's/Architecture: //')      section=$(grep -m1 'Section: ' info.txt | sed -e 's/Section: //' -e 's/\//_/g')      file="${arch}_${section}_packages.txt"      echo $p >> "package_files/$file"  done <manually_installed.txt    rm info.txt  


Solution:13

#! /bin/sh  DEFAULT=`tempfile`  CURRENT=`tempfile`  cat /var/log/installer/initial-status.gz | gzip -d | grep '^Package:' | awk '{ print $2}' | sort -u > $DEFAULT  aptitude search '~i !~M' -F '%p' | sort -u | tr -d ' ' | awk '{ print $1}' > $CURRENT  comm -23 $CURRENT $DEFAULT  rm $DEFAULT  rm $CURRENT  


Solution:14

I didn't find one that was suitable for me, so I wrote a script. By analysis of the contents of /var/log/apt/history.log* and /var/log/aptitude, then sort by datetime, resulting in two lists, one is manually installed pkgs, another is installed but removed pkgs. Script with 'apt list' verification. a little deviation, because the use of dpkg installed and modified PPA source.

I put here, https://github.com/eexpress/eexp-bin/blob/master/self-installed-pkg.pl.

if anyone need test it, maybe need modify some code, for my log files are mixed in Chinese and English (specially RFC2822 datetime format), I need deal with it. enter image description here


Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Previous
Next Post »