Emacs package install script in init file

Question

I restructured my Emacs init files to put them under version control. The file system layout looks like that:

/home/axel/.user_config/emacs
├── development.el
├── general.el
├── init.el
├── packages.el
└── writing.el

packages.el contains a list of packages Emacs should install on startup if they are not already installed. This works without problems after I do a rm -rf ~/.emacs.d/*. When I start Emacs afterwards, all the packages listed in packages.el are installed. However, when I manually add a package to (e.g. markdown-mode+) to the list pfl-packages in packages.el the new package is not installed when I restart Emacs. Deleting the contents of ~/.emacs.d/ again solves that problem, but I would like to be able to add package names to the list, which Emacs then automatically installs on startup. Am I missing something crucial about the Emacs initialization process?

Please see the content of the relevant files below.

The file ~/.emacscontains the following:

(add-to-list 'load-path "~/.user_config/emacs/")
(load-library "init")

The file ~/.user_config/emacs/init.el contains:

;;;; loads the different libraries to set up Emacs accordingly

;; load the library that sets up package repositories and syncing
(load-library "packages")

;; load the library that sets up the general behavior of Emacs
(load-library "general")
(load-library "writing")
(load-library "development")

The file packages.el contains the following:

;;;; this library sets up package repositories and allows for syncing    packages
;;;; between different machines
;;;; to add packages to the syncing, add their repository names to the   list `pfl-packages'

;;; set up package repositories from which additional packages can be installed

;; set up ELPA and MELPA repositories
(package-initialize)
(add-to-list 'package-archives
                     '("melpa" . "http://melpa.org/packages/"))
(add-to-list 'package-archives
                     '("org" . "http://orgmode.org/elpa/"))


;;; set up package syncing to allow for syncing between different machines

;; list of packages to sync
(setq pfl-packages
        '(
            color-theme-sanityinc-solarized
            company
            company-auctex
            company-c-headers
            company-irony
            company-quickhelp
            elpy
            ess
            flycheck-irony
            irony
            magit
            markdown-mode
            markdown-mode+
            rainbow-delimiters
            smart-tabs-mode
            yasnippet
            ))

;; refresh package list if it is not already available
(when (not package-archive-contents) (package-refresh-contents))

;; install packages from the list that are not yet installed
(dolist (pkg pfl-packages)
    (when (and (not (package-installed-p pkg)) (assoc pkg package-archive-contents))
        (package-install pkg)))

Answer 1

In my ~/.emacs.d directory, I have in my init.el the following:

(load "~/.emacs.d/init-packages")

In init-packages.el I have done exactly what you want to do: upon startup, emacs checks whether every package listed is already installed, and, if not, it is installed. When you want a new package installed, you just add it to the list.

It's worth noting that package-initialize has to be called after adding the relevant package archives, so that they can actually be fetched and then installed.

Below is the contents of init-packages.el:

(require 'package)

(add-to-list 'package-archives
             '("elpy" . "http://jorgenschaefer.github.io/packages/"))

(add-to-list 'package-archives
             '("marmalade" . "http://marmalade-repo.org/packages/"))

(add-to-list 'package-archives
             '("melpa-stable" . "http://melpa-stable.milkbox.net/packages/") t)

(add-to-list 'load-path "~/.emacs.d/site-lisp/")


; list the packages you want
(setq package-list
    '(python-environment deferred epc 
        flycheck ctable jedi concurrent company cyberpunk-theme elpy 
        yasnippet pyvenv highlight-indentation find-file-in-project 
        sql-indent sql exec-path-from-shell iedit
        auto-complete popup let-alist magit git-rebase-mode 
        git-commit-mode minimap popup))


; activate all the packages
(package-initialize)

; fetch the list of packages available 
(unless package-archive-contents
  (package-refresh-contents))

; install the missing packages
(dolist (package package-list)
  (unless (package-installed-p package)
    (package-install package)))

Hope this is what you were looking for!