pythonpandasfinanceback-testing

backtesting.py finding the best values for Annual Return, SQN, Win Rate, Final Equity or other


Backtesting.py produces an object of the following type:backtesting._stats._Stats. I'm using three different methods (RSI, 4 moving averages, skopt) to determine Annual Return, SQN, Win Rate, Final Equity in three different instances of stats. stats_RSI, stats_Sma4, stats_skopt. I would like to reference Annual Return, SQN, Win Rate, Final Equity and have a summary of which method produces the best result for each (highest values). Now I do this empirically:

#Comparison of all methods 
print('\033[1m' + 'Method 1: RSI' + '\033[0m''\nAnnual Return: ',round(stats_RSI["Return (Ann.) [%]"],2),
  '\nSortino Ratio: ',round(stats_RSI["Sortino Ratio"],2), '\nSQN: ',
  round(stats_RSI["SQN"],2),'\nWin Rate: ',round(stats_RSI["Win Rate [%]"],2),
  '\nFinal Equity: ',round(stats_RSI["Equity Final [$]"],2))

print('\033[1m' + '\nMethod 2: Moving Averages'+ '\033[0m''\nAnnual Return: ',round(stats_Sma4["Return (Ann.) [%]"],2),
  '\nSortino Ratio: ',round(stats_Sma4["Sortino Ratio"],2), '\nSQN: ',
  round(stats_Sma4["SQN"],2),'\nWin Rate: ',round(stats_Sma4["Win Rate [%]"],2),
  '\nFinal Equity: ',round(stats_Sma4["Equity Final [$]"],2))

print('\033[1m' + '\nMethod 3: Scikit Optimize'+ '\033[0m''\nAnnual Return: ',round(stats_skopt["Return (Ann.) [%]"],2),
  '\nSortino Ratio: ',round(stats_skopt["Sortino Ratio"],2), '\nSQN: ',
  round(stats_skopt["SQN"],2),'\nWin Rate: ',round(stats_skopt["Win Rate [%]"],2),
  '\nFinal Equity: ',round(stats_skopt["Equity Final [$]"],2))

which produces:

Method 1: RSI
Annual Return:  16.82 
Sortino Ratio:  1.06 
SQN:  2.77 
Win Rate:  82.35 
Final Equity:  37628.0

Method 2: Moving Averages
Annual Return:  4.23 
Sortino Ratio:  0.35 
SQN:  0.73 
Win Rate:  41.27 
Final Equity:  14240.32

Method 3: Scikit Optimize
Annual Return:  13.73 
Sortino Ratio:  1.25 
SQN:  1.69 
Win Rate:  47.62 
Final Equity:  29941.38

I would like to automatically reference the best value for each category with a reference to the method used for obtaining that value. Each method is saved in its own object (stats_RSI,stats_Sma4,stats_skopt). I have looked at compare(), and max() but I'm not sure how to integrate to obtain best values for each category with a reference to the method used. The idea is to then let the human pick the method according to the value central to a given investment strategy.

Here is what backtesting produces for stats:

Start                     2004-08-19 00:00:00
End                       2013-03-01 00:00:00
Duration                   3116 days 00:00:00
Exposure Time [%]                   38.500931
Equity Final [$]                    29941.377
Equity Peak [$]                   35721.40136
Return [%]                          199.41377
Buy & Hold Return [%]              703.458242
Return (Ann.) [%]                    13.73011
Volatility (Ann.) [%]               19.395026
Sharpe Ratio                         0.707919
Sortino Ratio                        1.254814
Calmar Ratio                         0.461043
Max. Drawdown [%]                  -29.780538
Avg. Drawdown [%]                   -3.864038
Max. Drawdown Duration     1942 days 00:00:00
Avg. Drawdown Duration       86 days 00:00:00
# Trades                                   42
Win Rate [%]                        47.619048
Best Trade [%]                      55.580827
Worst Trade [%]                     -5.446038
Avg. Trade [%]                       2.684772
Max. Trade Duration         111 days 00:00:00
Avg. Trade Duration          28 days 00:00:00
Profit Factor                        3.486524
Expectancy [%]                       3.170025
SQN                                    1.6908
_strategy                 Sma4Cross(n1=44,...
_equity_curve                            E...
_trades                       Size  EntryB...
dtype: object

I would like to obtain something like:

Annual Return: 16.82 stats_rsi
Sortino Ratio: 1.25 stats_skopt
SQN: 2.77 stats_rsi
Win Rate: 82.35 stats_rsi
Final Equity: 37628.0 stats_rsi

EDIT

I did this:

methods = [stats_RSI, stats_Sma4, stats_skopt]
annual_return = 0
sortino_ratio = 0
sqn = 0
win_rate = 0
final_equity = 0
for method in methods:
   if method["Return (Ann.) [%]"] > annual_return:
       annual_return = method["Return (Ann.) [%]"]
       print("Annual Return:\n", annual_return, "\n", method._strategy, "\n")
   if method["Sortino Ratio"] > sortino_ratio:
        sortino_ratio = method["Sortino Ratio"]
        print("Sortino Ratio:\n", sortino_ratio, "\n", method._strategy, "\n")
   if method["SQN"] > sqn:
       sqn = method["SQN"]
       print("SQN:\n", sqn, "\n", method._strategy, "\n")
   if method["Win Rate [%]"] > win_rate:
       win_rate = method["Win Rate [%]"]
       print("Win Rate:\n", win_rate, "\n", method._strategy, "\n")
   if method["Equity Final [$]"] > final_equity:
       final_equity = method["Equity Final [$]"]
        print("Final Equity:\n", final_equity, "\n", method._strategy, "\n")

Obtaining:
Annual Return:
16.820232474460074.
RsiOscillator(upper_bound=70,lower_bound=35,rsi_window=14).

Sortino Ratio:
1.058212866533837.
RsiOscillator(upper_bound=70,lower_bound=35,rsi_window=14)

SQN:
2.7697514727638506.
RsiOscillator(upper_bound=70,lower_bound=35,rsi_window=14).

Win Rate:
82.35294117647058.
RsiOscillator(upper_bound=70,lower_bound=35,rsi_window=14).

Final Equity:
37628.0.
RsiOscillator(upper_bound=70,lower_bound=35,rsi_window=14).

Sortino Ratio:
1.2548141815003493.
Sma4Cross(n1=44,n2=134,n_enter=39,n_exit=27).

but I'm sure there's a better way with less lines of code...

EDIT 2 See slothrop's answer below (thanks). His approach is environment friendly with an execution time of 863 µs whereas mine took 15.5 ms to execute. This community is very special and I appreciate all the talent animating its generosity and willingness to share knowledge.


Solution

  • How about something like this?

    result_objs = {'RSI': stats_RSI, 'SMA4': stats_SMA4, 'SKOpt': stats_skopt}
    metrics = ['annual_return', 'sortino_ratio', 'sqn', 'win_rate', 'final_equity']
    
    for metric in metrics:
       results_by_method = {k: getattr(v, metric) for k, v in result_objs.items()}
       best_method, best_result = max(results_by_method.items(), key=lambda t: t[-1])
       print(f'{metric}: {best_result} (from {best_method})')