How to use Stuart Sierra's component library in Clojure [closed]

Question

I'm struggling to get my head around how to use Stuart Sierra's component library within a Clojure app. From watching his Youtube video, I think I've got an OK grasp of the problems that led to him creating the library; however I'm struggling to work out how to actually use it on a new, reasonably complex project.

I realise this sounds very vague, but it feels like there's some key concept that I'm missing, and once I understand it, I'll have a good grasp on how to use components. To put it another way, Stuart's docs and video go into the WHAT and WHY of components in considerable detail, but I'm missing the HOW.

Is there any sort of detailed tutorial/walkthrough out there that goes into:

• why you'd use components at all for a non-trivial Clojure app
• a methodology for how you'd break down the functionality in a non-trivial Clojure app, such that components can be implemented in a reasonably optimal fashion. It's reasonably simple when all you've got is e.g. a database, an app server and a web server tier, but I'm struggling to grasp how you'd use it for a system that has many different layers that all need to work together coherently
• ways to approach development/testing/failover/etc. in a non-trivial Clojure app that's been built using components

Thanks in advance

Answer 1

In short, Component is a specialized DI framework. It can set up an injected system given two maps: the system map and the dependency map.

Let's look at a made-up web app (disclaimer, I typed this in a form without actually running it):

(ns myapp.system
  (:require [com.stuartsierra.component :as component]
            ;; we'll talk about myapp.components later
            [myapp.components :as app-components]))

(defn system-map [config] ;; it's conventional to have a config map, but it's optional
  (component/system-map
    ;; construct all components + static config
    {:db (app-components/map->Db (:db config))
     :handler (app-components/map->AppHandler (:handler config))
     :server (app-components/map->Server (:web-server config))}))

(defn dependency-map
  ;; list inter-dependencies in either:
  ;;    {:key [:dependency1 :dependency2]} form or
  ;;    {:key {:name-arg1 :dependency1
  ;;           :name-arg2 :dependency2}} form
  {:handler [:db]
   :server {:app :handler})

;; calling this creates our system
(def create-system [& [config]]
  (component/system-using
    (system-map (or config {})
    (dependency-map)))

This allows us to call (create-system) to create a new instance of our entire application when we need one.

Using (component/start created-system), we can run a system's services it provides. In this case, it's the webserver that's listening on a port and an open db connection.

Finally, we can stop it with (component/stop created-system) to stop the system from running (eg - stop the web server, disconnect from db).

Now let's look at our components.clj for our app:

(ns myapp.components
  (:require [com.stuartsierra.component :as component]
            ;; lots of app requires would go here
            ;; I'm generalizing app-specific code to
            ;; this namespace
            [myapp.stuff :as app]))

(defrecord Db [host port]
   component/Lifecycle
   (start [c]
      (let [conn (app/db-connect host port)]
        (app/db-migrate conn)
        (assoc c :connection conn)))
   (stop [c]
      (when-let [conn (:connection c)]
        (app/db-disconnect conn))
      (dissoc c :connection)))

(defrecord AppHandler [db cookie-config]
   component/Lifecycle
   (start [c]
      (assoc c :handler (app/create-handler cookie-config db)))
   (stop [c] c))

;; you should probably use the jetty-component instead
;; https://github.com/weavejester/ring-jetty-component
(defrecord Server [app host port]
   component/Lifecycle
   (start [c]
      (assoc c :server (app/create-and-start-jetty-server
                        {:app (:handler app)
                         :host host 
                         :port port})))
   (stop [c]
      (when-let [server (:server c)]
         (app/stop-jetty-server server)
      (dissoc c :server)))

So what did we just do? We got ourselves a reloadable system. I think some clojurescript developers using figwheel start seeing similarities.

This means we can easily restart our system after we reload code. On to the user.clj!

(ns user
    (:require [myapp.system :as system]
              [com.stuartsierra.component :as component]
              [clojure.tools.namespace.repl :refer (refresh refresh-all)]
              ;; dev-system.clj only contains: (def the-system)
              [dev-system :refer [the-system]])

(def system-config
  {:web-server {:port 3000
                :host "localhost"}
   :db {:host 3456
        :host "localhost"}
   :handler {cookie-config {}}}

(def the-system nil)

(defn init []
  (alter-var-root #'the-system
                  (constantly system/create-system system-config)))

(defn start []
  (alter-var-root #'the-system component/start))

(defn stop []
  (alter-var-root #'the-system
                  #(when % (component/stop %))))

(defn go []
  (init)
  (start))

(defn reset []
  (stop)
  (refresh :after 'user/go))

To run a system, we can type this in our repl:

(user)> (reset)

Which will reload our code, and restart the entire system. It will shutdown the exiting system that is running if its up.

We get other benefits:

• End to end testing is easy, just edit the config or replace a component to point to in-process services (I've used it to point to an in-process kafka server for tests).
• You can theoretically spawn your application multiple times for the same JVM (not really as practical as the first point).
• You don't need to restart the REPL when you make code changes and have to restart your server
• Unlike ring reload, we get a uniform way to restart our application regardless of its purpose: a background worker, microservice, or machine learning system can all be architected in the same way.

It's worth noting that, since everything is in-process, Component does not handle anything related to fail-over, distributed systems, or faulty code ;)

There are plenty of "resources" (aka stateful objects) that Component can help you manage within a server:

• Connections to services (queues, dbs, etc.)
• Passage of Time (scheduler, cron, etc.)
• Logging (app logging, exception logging, metrics, etc.)
• File IO (blob store, local file system, etc.)
• Incoming client connections (web, sockets, etc.)
• OS Resources (devices, thread pools, etc.)

Component can seem like overkill if you only have a web server + db. But few web apps are just that these days.

Side Note: Moving the-system into another namespace reduces the likelihood of refreshing the the-system var when developing (eg - calling refresh instead of reset).