I am calculating Anderson Darling k-sample statistic for every variable in a dataframe with a subset of that dataframe and adding the significance level as text to plots generated with a loop. The test generates a lengthy "UserWarning" for every calculation that is being printed before my plots get generated.
Is there a way to suppress "UserWarning" for only a single line of code.
For example, R has a function suppressWarnings()
that suppresses the warnings for code within the parenthesis. I could have passed suppressWarnings(stats.anderson_ksamp([a,b])
in R to ensure no UserWarning shows up.
from scipy import stats
a = np.random.normal(size=50)
b = np.random.normal(loc=0.5, size=30)
stats.anderson_ksamp([a,b])
Output:
<ipython-input-91-91b6d1abc8ed>:1: UserWarning: p-value floored: true value smaller than 0.001
stats.anderson_ksamp([np.random.normal(size=50), np.random.normal(loc=0.5, size=30)])
Out[91]: Anderson_ksampResult(statistic=7.638735956038192, critical_values=array([0.325, 1.226, 1.961, 2.718, 3.752, 4.592, 6.546]), significance_level=0.001)
When I use this code in a loop, it generates multiple such warnings for each calculation.
Is there a way to suppress these UserWarnings
a) with a function : very_useful_suppressor_function(stats.anderson_ksamp([a,b]))
or
b) a way to suppress warnings in 1 jupyter notebook cell
or
c) any other way
To recreate the issue with random numbers:_
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
from scipy.stats import anderson_ksamp
from scipy.stats import ks_2samp
a = np.random.normal(size=50)
b = np.random.normal(loc=0.5, size=30)
stats.anderson_ksamp([a,b])
#%%%
college = pd.DataFrame(np.random.randint(1,1000, size=(10,10)))
college
college[0:5]
#%%%
## Boxplots
subset_df = college[0:5]
base_df = college
var_main = 0
ovars = np.setdiff1d(subset_df.columns, var_main)
n = len(ovars)
ovars
#%%%
for var in ovars:
fig, (ax1, ax2) = plt.subplots(nrows=2, sharex=True, figsize=(12,2), facecolor='w')
s1 = base_df[var]
s2 = subset_df[var]
adt = anderson_ksamp([s1,s2], midrank=True)
kst = ks_2samp(s1,s2, 'two-sided')
plt.figtext(0, 1, 'AD test: sig-lvl ='+str(round(adt[2],5)))
plt.figtext(0, 0.9, 'KS test: p-val ='+str(kst[1].round(5)))
ax1 = plt.subplot(211)
sns.boxplot(x=subset_df[var], width=0.4, color='thistle', fliersize=3)
plt.axis('off')
ax1.set(title = '-'*20+'\n'+str(var))
ax2 = plt.subplot(212, sharex=ax1)
box = sns.boxplot(x=base_df[var], width=0.5, color='cadetblue', fliersize=3)
ax2.set(xlabel=None)
ax2.tick_params(left=False);
Got the plots after all this waste of space:
C:\Users\Rahul\.spyder-py3\temp.py:39: UserWarning: p-value capped: true value larger than 0.25
adt = anderson_ksamp([s1,s2], midrank=True)
C:\Users\Rahul\.spyder-py3\temp.py:39: UserWarning: p-value capped: true value larger than 0.25
adt = anderson_ksamp([s1,s2], midrank=True)
C:\Users\Rahul\.spyder-py3\temp.py:39: UserWarning: p-value capped: true value larger than 0.25
adt = anderson_ksamp([s1,s2], midrank=True)
C:\Users\Rahul\.spyder-py3\temp.py:39: UserWarning: p-value capped: true value larger than 0.25
adt = anderson_ksamp([s1,s2], midrank=True)
C:\Users\Rahul\.spyder-py3\temp.py:39: UserWarning: p-value capped: true value larger than 0.25
adt = anderson_ksamp([s1,s2], midrank=True)
C:\Users\Rahul\.spyder-py3\temp.py:39: UserWarning: p-value capped: true value larger than 0.25
adt = anderson_ksamp([s1,s2], midrank=True)
C:\Users\Rahul\.spyder-py3\temp.py:39: UserWarning: p-value capped: true value larger than 0.25
adt = anderson_ksamp([s1,s2], midrank=True)
C:\Users\Rahul\.spyder-py3\temp.py:39: UserWarning: p-value capped: true value larger than 0.25
adt = anderson_ksamp([s1,s2], midrank=True)
C:\Users\Rahul\.spyder-py3\temp.py:39: UserWarning: p-value capped: true value larger than 0.25
adt = anderson_ksamp([s1,s2], midrank=True)
There is a way to do so with with
statement.
source
### a) Define function to suppress warnings
import warnings
## For UserWarning
def fxnUw():
warnings.warn("UserWarning arose", UserWarning)
## For DeprecationWarning
def fxnDw():
warnings.warn("deprecated", DeprecationWarning)
### b) with statement to suppress
with warnings.catch_warnings():
warnings.simplefilter("ignore")
fxnUw()
# code starts here #