clojurefunctional-programmingclojurescriptclojure.specgenerative-testing

What is generative testing in Clojure?


I came across Generative Testing in Clojure with spec notion and would like to learn about it.

Also providing some examples would be very useful.


Solution

  • As introductory reading we've got the Rationale and Overview along with the Guide which should provide you with information both about the why and the how.

    If you'd like a somewhat complex example, we can take the string->semantic-version function of leiningen.release:

    (defn string->semantic-version [version-string]
      "Create map representing the given version string. Returns nil if the
      string does not follow guidelines setforth by Semantic Versioning 2.0.0,
      http://semver.org/"
      ;; <MajorVersion>.<MinorVersion>.<PatchVersion>[-<Qualifier>][-SNAPSHOT]
      (if-let [[_ major minor patch qualifier snapshot]
               (re-matches
                #"(\d+)\.(\d+)\.(\d+)(?:-(?!SNAPSHOT)([^\-]+))?(?:-(SNAPSHOT))?"
                version-string)]
        (->> [major minor patch]
             (map #(Integer/parseInt %))
             (zipmap [:major :minor :patch])
             (merge {:qualifier qualifier
                     :snapshot snapshot}))))
    

    It takes a string and tries to parse it into a program-readable map representing the version number of some artifact. A spec for it could look like:

    First some dependencies

    (ns leiningen.core.spec.util
      (:require
       [clojure.spec           :as spec]
       [clojure.spec.gen       :as gen]
       [miner.strgen           :as strgen]
       [clojure.spec.test      :as test]
       [leiningen.release      :as release]))
    

    then a helper macro

    (defmacro stregex
      "Defines a spec which matches a string based on a given string
      regular expression. This the classical type of regex as in the
      clojure regex literal #\"\""
      [string-regex]
      `(spec/with-gen
         (spec/and string? #(re-matches ~string-regex %))
         #(strgen/string-generator ~string-regex)))
    

    followed by a definition of a semantic version

    (spec/def ::semantic-version-string
      (stregex #"(\d+)\.(\d+)\.(\d+)(-\w+)?(-SNAPSHOT)?"))
    

    and some helper-specs

    (spec/def ::non-blank-string
      (spec/and string? #(not (str/blank? %))))
    (spec/def ::natural-number
      (spec/int-in 0 Integer/MAX_VALUE))
    

    for the definition of the keys in the resulting map

    (spec/def ::release/major     ::natural-number)
    (spec/def ::release/minor     ::natural-number)
    (spec/def ::release/patch     ::natural-number)
    (spec/def ::release/qualifier ::non-blank-string)
    (spec/def ::release/snapshot  #{"SNAPSHOT"})
    

    and the map itself

    (spec/def ::release/semantic-version-map
      (spec/keys :req-un [::release/major ::release/minor ::release/patch
                          ::release/qualifier ::release/snapshot]))
    

    followed by the function spec:

    (spec/fdef release/string->semantic-version
               :args (spec/cat :version-str ::release/semantic-version-string)
               :ret  ::release/semantic-version-map)
    

    By now we can let Clojure Spec generate test data and feed it into the function itself in order to test whether it meets the constraints we've put up for it:

    (test/check `release/version-map->string)
    => ({:spec #object[clojure.spec$fspec_impl$reify__14248 0x16c2555 "clojure.spec$fspec_impl$reify__14248@16c2555"],
         :clojure.spec.test.check/ret {:result true,
                                       :num-tests 1000,
                                       :seed 1491922864713},
         :sym leiningen.release/version-map->string})
    

    This tells us that out of the 1000 test cases spec generated for us the function passed every single one.