clojureclojure-testinggenerative-testing

How to generative test function-under-test with must-to-test distinct edge cases for each execution?


Hello Clojure experts..!,

Update: 08/03/2018 1:25PM CDT - Rephrased the question to make it more descriptive.

Update: 08/03/2018 4:10PM CDT - Added negative scenario test to make more sense.

I have a function/spec-under-test (jvm-languages), i want to test positive scenario(for now) using Clojure generative test API.

Positive scenario test: Given the jvm-language as one of the five {"clojure" "haskell" "erlang" "scala" "python"}, result should be true.

My Question/requirement: I should want to test my function-under-test with each and every possible valid scenario (which i listed in a set below) for each execution (each time i run, lein test) so that i can get rid of regular unit tests.

As per my little testing knowledge, i think there is no value(for me) to test my function multiple times with the same test-scenario in the same execution. Please educate me, if you think my thought process is wrong.

  1. Is there a way to fulfill my use-case using Clojure generative testing?
  2. If not, is it not a great feature to have?

       (ns spec.gen-test.explore
    (:require [clojure.test                    :refer :all]
          [clojure.spec                    :as spec]
          [clojure.test.check.generators   :as gen]
          [clojure.test.check.properties   :as prop :include-macros true]
          [clojure.test.check.clojure-test :refer [defspec]]))
    (def languages-set #{"clojure" "haskell" "erlang" "scala" "python"})
    
    (spec/def ::jvm-languages 
    (spec/and string?
     languages-set))
    
    ;;Generative test case for jvm-languages spec:
    (defspec jvm-languages-positive-generative-test
     5
     (prop/for-all [language (gen/elements ["clojure" "haskell" "erlang" 
     "scala" "python"])]
                (println "Testing for language:" language)
                (spec/valid? ::jvm-languages language)))
    
    (defspec jvm-languages-negative-generative-test
      100
      (prop/for-all [language (gen/such-that
                               #(not (contains? languages-set %))
                               (gen/one-of
                                 [ gen/string
                                   (gen/elements ["" nil])
                                   gen/int
                                   gen/keyword
                                   gen/char-alphanumeric
                                   gen/double ]))]
                (println "Testing for language:" language)
                (not (spec/valid? ::jvm-languages language))))
    (run-tests)
    
    Output: 
    #'spec.gen-test.explore/jvm-languages-positive-generative-test
    Testing for language: scala
    Testing for language: haskell
    Testing for language: erlang
    Testing for language: erlang
    Testing for language: scala
    {:result true, :num-tests 5, :seed 1533232724897, :test-var "jvm-languages-generative-test"}
    

Solution

  • I'd just iterate the collection since you want to test each case anyway.

    You could get some reasonably nice-reading code using is and every?:

    (ns your-ns
      (:require [clojure.test :refer [is testing]]))
    
    (let [test-cases ["clojure" "haskell" "erlang" "scala" "python"]]
      (testing "if my things are working."
        ; Just iterating the test cases using every?
        (is (every? #(your-test-predicate-here %) test-cases))))