unit-testingclojuregenerative-testing

Howto include clojure.spec'd functions in a test suite


Is there anyway to include clojure.spec'd functions in a generalized test suite? I know we can register specs and directly spec functions.

(ns foo
  (:require [clojure.spec :as s]
            [clojure.spec.test :as stest]))

(defn average [list-sum list-count]
  (/ list-sum list-count))

(s/fdef average
        :args (s/and (s/cat :list-sum float? :list-count integer?)
                     #(not (zero? (:list-count %))))
        :ret number?)

And later, if I want to run generative tests against that spec'd function, I can use stest/check.

=> (stest/check `average)
({:spec #object[clojure.spec$fspec_impl$reify__14282 0x68e9f37c "clojure.spec$fspec_impl$reify__14282@68e9f37c"], :clojure.spec.test.check/ret {:result true, :num-tests 1000, :seed 1479587517232}, :sym edgar.core.analysis.lagging/average})

But i) is there anyway to include these test runs in my general test suite? I'm thinking of the kind of clojure.test integration that test.check has. The closest thing that I can see ii) is the stest/instrument (see here) function. But that seems to just let us turn on checking at the repl. Not quite what I want. Also, iii) are function specs registered?

(defspec foo-test 
         100 

         ;; NOT this
         #_(prop/for-all [v ...]
           (= v ...))

         ;; but THIS
         (stest/some-unknown-spec-fn foo))

Solution

  • Ok, solved this one. Turns out there's no solution out of the box. But some people on the clojure-spec slack channel have put together a defspec-test solution for clojure.spec.test and clojure.test.

    So given the code in the question. You can A) define the defspec-test macro that takes your test name and a list of spec'd functions. You can then B) use it in your test suite.

    Thanks Clojure community!! And hopefully such a utility function makes it into the core library.

    A)

    (ns foo.test
      (:require [clojure.test :as t]
                [clojure.string :as str]))
    
    (defmacro defspec-test
      ([name sym-or-syms] `(defspec-test ~name ~sym-or-syms nil))
      ([name sym-or-syms opts]
       (when t/*load-tests*
         `(def ~(vary-meta name assoc
                           :test `(fn []
                                    (let [check-results# (clojure.spec.test/check ~sym-or-syms ~opts)
                                          checks-passed?# (every? nil? (map :failure check-results#))]
                                      (if checks-passed?#
                                        (t/do-report {:type    :pass
                                                      :message (str "Generative tests pass for "
                                                                    (str/join ", " (map :sym check-results#)))})
                                        (doseq [failed-check# (filter :failure check-results#)
                                                :let [r# (clojure.spec.test/abbrev-result failed-check#)
                                                      failure# (:failure r#)]]
                                          (t/do-report
                                            {:type     :fail
                                             :message  (with-out-str (clojure.spec/explain-out failure#))
                                             :expected (->> r# :spec rest (apply hash-map) :ret)
                                             :actual   (if (instance? Throwable failure#)
                                                         failure#
                                                         (:clojure.spec.test/val failure#))})))
                                      checks-passed?#)))
            (fn [] (t/test-var (var ~name)))))))
    

    B)

    (ns foo-test
      (:require [foo.test :refer [defspec-test]]
                [foo]))
    
    
    (defspec-test test-average [foo/average])