What/Where are the final threshold(s) used to score new predictions during deployment in DAI?
I have a binary classification problem, ran it through DAI, and am given test AUC's. Where do I find the probability threshold that is used during deployment to score new rows of data?
An example would be a threshold of .50; I.E a target variable >.50 gets a 1, and target<.50 gets a 0 (or vice versa) during a decision. I need the exact threshold beyond the 4 digit concatenated number that is shown in the GUI as you move across the AUC curve. In the pictures below I've matched the thresholds and can't get the same confusion matrices with the identical threshold. Notice the False Positives are very minimally different.
DAI Threshold @ .0301 Threshold
Sklearn Confusion Matrix @ .0301 Threshold
UPDATED ANSWER: Download the "Experiments Summary" tab after a completed experiment on DAI. Within the zip file you'll find an ensemble_roc_test.json that gives thresholds up to 10 digits.
Driverless AI predictions return 'scores', so thresholding is not applied to the prediction and it is up to the user how they use the score. It is possible to see the recommended threshold to optimize different metrics on the experiment page ROC curve. For example, in the screenshot below the mouse is hovering over the 'Best F1' circle to get a summary that includes the threshold:
- count the elements in a list
- How to solve: ValueError: Invalid format specifier?
- How to generate an MD5 checksum of a file?
- What is the point for asyncio synchronization primitives not to be thread safe?
- How to increase a counter in SQLAlchemy
- How update/remove conda-forge channel from Anaconda?
- Editing specific labels in stacked bar chart with matplotlib
- Strange results with huggingface transformer[marianmt] translation of larger text
- Appending column values from one dataframe to another as a list
- Intersection 3D meshes python
- extract variables from list and run a loop
- Overlay polygon on top of image in Python
- bit-wise operation unary ~ (invert)
- Variable table name in sqlite
- SQL SELECT from table name supplied by a variable
- How do I detect collision in pygame?
- Can you override the default formatter for f-strings?
- Streaming function in Microsoft Autogen
- Importing dynamically all modules from a folder
- Is it possible to type annotate Python function parameter used as TypedDict key to make mypy happy?
- Using VirtualEnv with multiple Python versions on windows
- How to pickle or store Jupyter (IPython) notebook session for later
- Why does a Pygame display with the pygame.FULLSCREEN flag affect other windows?
- color discrete sequence in a plotly graph objects
- How to specify dependencies for the entire router?
- What is the easiest way to share a python environment?
- docker streamlink opens a stream but immediately closes it
- How to attach a Scrollbar to a Text widget?
- Getting today's date in YYYY-MM-DD in Python?
- Get the current git hash in a Python script