pythonpandasp-valuepearson-correlation

How to add adjusted pvalue to the correlation analysis?


I want to add adj.pval to the correlation matrix generated using pearson. Below are my codes to generate the pairwise matrix.

from pandas import *
import numpy as np
import pandas as pd
from scipy.stats.stats import pearsonr
import itertools

# Generate sample data
data = np.random.rand(10, 6)

# Create a DataFrame with row names and column names
genes = ['Gene{}'.format(i+1) for i in range(6)]
samples = ['Sample{}'.format(i+1) for i in range(10)]

df = pd.DataFrame(data, index=samples, columns=genes)

correlations = {}
#store the gene names
columns = df.columns.tolist()

#perform pearson correlation
for col_a, col_b in itertools.combinations(columns, 2):
    correlations[col_a +':' + col_b] = pearsonr(df.loc[:, col_a], df.loc[:, col_b])

#Convert the dictionary into dataframe
result = DataFrame.from_dict(correlations, orient='index')
result.columns = ['PCC', 'p-value']
#Convert index as a column
result.reset_index(level=0, inplace=True)
#rename the converted column
result.rename(columns = {'index':'Genes'}, inplace = True)
#split the column as two columns
result[['Gene-1','Gene-2']] = result.Genes.str.split(":", expand=True)
#select the required variables
result = result[['Gene-1', 'Gene-2', 'PCC', 'p-value']]

output

It gave me the pairwise matrix with pvalue. I want to add adj.pval as well? How do I add it?


Solution

  • You can use multitest to calculate adjusted p value

    from statsmodels.stats import multitest
    # Calculate adjusted p-values
    reject, adj_pvals, _, _ = multitest.multipletests(result['p-value'], method='bonferroni')
    
    # Add adjusted p-value to the DataFrame
    result['adj.pval'] = adj_pvals