vectorbt学习_51DMA之十一滑窗网格参数优选

本文在上一篇文章(31DMA之九滑窗网格参数优选)基础上。
之前文章:
01,增加了止盈，止损，跟踪止损等参数，但实际效果看训练集上效果尚可，验证集上效果更差，怀疑过拟合导致。
02.增加几种避免过拟合的参数优选方法。
新增3种参数优选方法，一定程度上降低参数过拟合的可能。

1
  v1:直接(简单最大值)优选法
2
  v2:邻近域优选法
3
  v3:邻居权重优选法-均值
4
  v4:邻居权重优选法-中位数

03,止损参数，止盈参数也是类似的，不止2个维度了，邻居采用立方体思路.对角点相接的也算作邻居，
之前2维时是同边才算邻居，比如(1,3)，邻居是(2,3),(1,2),(1,4),新的规则会新增(2，2),(2，4)

本文增加：增加行情过滤器，过滤掉低波动行情，开仓时如果行情波动性不足，不开仓。
原始买卖状态信号：
1111111100000000000001111100
=》+ 过滤器
0011100000000000000111100000
=》期望效果
0011111100000000000001111100

可见过滤器影响原始买卖状态信号的头，不影响原始买卖状态的尾巴。

01,基础配置信息#

1
#conda envs:vectorbt_env
2
import warnings
3
import vectorbt as vbt
4
import numpy as np
5
import pandas as pd
6
from datetime import datetime, timedelta
7
import pytz
8
from dateutil.parser import parse
9
import ipywidgets as widgets
10
from copy import deepcopy
11
from tqdm import tqdm
12
import imageio
13
from IPython import display
14
import plotly.graph_objects as go
15
import itertools
16
import dateparser
17
import gc
18
import math
19
from tools import dbtools
20

21
warnings.filterwarnings("ignore")
22

23
pd.set_option('display.max_rows',500)
24
pd.set_option('display.max_columns',500)
25
pd.set_option('display.width',1000)

02,行情获取和可视化#

a,时间交易参数配置#

1
# Enter your parameters here
2
seed = 42
3
symbol = '002594.XSHE'
4
metric = 'total_return'
5

6
start_date = datetime(2020, 1, 1, tzinfo=pytz.utc)  # time period for analysis, must be timezone-aware
7
end_date = datetime(2023,1,1, tzinfo=pytz.utc)
8
time_buffer = timedelta(days=100)  # buffer before to pre-calculate SMA/EMA, best to set to max window
9
freq = '1D'
10

11
vbt.settings.portfolio['init_cash'] = 10000.  # 100$
12
vbt.settings.portfolio['fees'] = 0.0025  # 0.25%
13
vbt.settings.portfolio['slippage'] = 0.0025  # 0.25%

b,获取行情和行情mask#

1
# Download data with time buffer
2
cols = ['Open', 'High', 'Low', 'Close', 'Volume']
3
# ohlcv_wbuf = vbt.YFData.download(symbol, start=start_date-time_buffer, end=end_date).get(cols)
4

5
ohlcv_wbuf=dbtools.MySQLData.download(symbol).get() # 自带工具类查询
6
assert(~ohlcv_wbuf.empty)
7
ohlcv_wbuf = ohlcv_wbuf.astype(np.float64)
8

9
print("ohlcv_wbuf.shape:",ohlcv_wbuf.shape)
10
print("ohlcv_wbuf.columns:",ohlcv_wbuf.columns)
11

12

13
# Create a copy of data without time buffer
14
wobuf_mask = (ohlcv_wbuf.index >= start_date) & (ohlcv_wbuf.index <= end_date) # mask without buffer
15

16
ohlcv = ohlcv_wbuf.loc[wobuf_mask, :]
17

18
print("ohlcv.shape:",ohlcv.shape)
19

20
# Plot the OHLC data
21
ohlcv.vbt.ohlcv.plot().show_svg() # 绘制蜡烛图
22
# remove show_svg() to display interactive chart!

1
ohlcv_wbuf.shape: (978, 5)
2
ohlcv_wbuf.columns: Index(['Open', 'High', 'Low', 'Close', 'Volume'], dtype='object')
3
ohlcv.shape: (728, 5)

svg

20,网格参数-指标计算和可视化#

仅可视化第一列

1
fast_windows = np.arange(10, 50,5)
2
slow_multis = np.arange(1.5, 5.5, 0.5)
3
print("fast_windows:",fast_windows)
4
print("slow_multis:",slow_multis)
5

6
price_wbuf=ohlcv_wbuf['Close']
7
dualma_wbuf = vbt.DualMA.run(price_wbuf, fast_window=fast_windows,slow_multi=slow_multis,param_product=True)
8
dualma = dualma_wbuf[wobuf_mask]
9
# there should be no nans after removing time buffer
10
assert(~dualma.fast_ma.isnull().any().any())
11
assert(~dualma.slow_ma.isnull().any().any())
12

13
# 计算收盘价的标准差
14
std_close_wbuf = ohlcv_wbuf['Close'].rolling(window=20).std()
15
std_close_ma_wbuf = std_close_wbuf.rolling(window=20).mean()
16
std_close=std_close_wbuf[wobuf_mask]
17
std_close_ma=std_close_ma_wbuf[wobuf_mask]
18
std_indicator = (std_close > std_close_ma )
19

20
# 计算收盘价diff
21
diff_close_wbuf = ohlcv_wbuf['Close'] - ohlcv_wbuf['Close'].rolling(window=int(20/5)).mean().shift(20)
22
diff_close_ma_wbuf = diff_close_wbuf.rolling(window=20).mean()
23
diff_close=diff_close_wbuf[wobuf_mask]
24
diff_close_ma=diff_close_ma_wbuf[wobuf_mask]
25
diff_indicator = ((diff_close > diff_close_ma )&(diff_close_ma>200*0.0025*20))
26

27
print()
28
print('dualma.fast_ma.head(3)')
29
print(dualma.fast_ma.head(3))
30
print('dualma.slow_ma.head(3)')
31
print(dualma.slow_ma.head(3))
32

33
print()
34
fig = ohlcv['Close'].vbt.plot(trace_kwargs=dict(name='Price'))
35
fig = dualma.fast_ma.iloc[:,0].vbt.plot(trace_kwargs=dict(name="Fast MA col %s"%str(dualma.fast_ma.iloc[:,0].name)), fig=fig)
36
fig = dualma.slow_ma.iloc[:,0].vbt.plot(trace_kwargs=dict(name="Slow MA col %s"%str(dualma.slow_ma.iloc[:,0].name)), fig=fig)
37
fig.show_svg()

1
fast_windows: [10 15 20 25 30 35 40 45]
2
slow_multis: [1.5 2.  2.5 3.  3.5 4.  4.5 5. ]
3

4
dualma.fast_ma.head(3)
5
dualma_fast_window             10                                                                 15                                                                                    20                                                                      25                                                                        30                                                                                      35                                                                                    40                                                                        45
6
dualma_slow_multi             1.5     2.0     2.5     3.0     3.5     4.0     4.5     5.0        1.5        2.0        2.5        3.0        3.5        4.0        4.5        5.0      1.5      2.0      2.5      3.0      3.5      4.0      4.5      5.0      1.5      2.0      2.5      3.0      3.5      4.0      4.5      5.0        1.5        2.0        2.5        3.0        3.5        4.0        4.5        5.0        1.5        2.0        2.5        3.0        3.5        4.0        4.5        5.0      1.5      2.0      2.5      3.0      3.5      4.0      4.5      5.0        1.5        2.0        2.5        3.0        3.5        4.0        4.5        5.0
7
date
8
2020-01-02 00:00:00+00:00  46.665  46.665  46.665  46.665  46.665  46.665  46.665  46.665  45.824667  45.824667  45.824667  45.824667  45.824667  45.824667  45.824667  45.824667  45.3025  45.3025  45.3025  45.3025  45.3025  45.3025  45.3025  45.3025  44.9476  44.9476  44.9476  44.9476  44.9476  44.9476  44.9476  44.9476  44.816667  44.816667  44.816667  44.816667  44.816667  44.816667  44.816667  44.816667  44.594571  44.594571  44.594571  44.594571  44.594571  44.594571  44.594571  44.594571  44.5425  44.5425  44.5425  44.5425  44.5425  44.5425  44.5425  44.5425  44.440222  44.440222  44.440222  44.440222  44.440222  44.440222  44.440222  44.440222
9
2020-01-03 00:00:00+00:00  46.972  46.972  46.972  46.972  46.972  46.972  46.972  46.972  46.128667  46.128667  46.128667  46.128667  46.128667  46.128667  46.128667  46.128667  45.5025  45.5025  45.5025  45.5025  45.5025  45.5025  45.5025  45.5025  45.1420  45.1420  45.1420  45.1420  45.1420  45.1420  45.1420  45.1420  44.964000  44.964000  44.964000  44.964000  44.964000  44.964000  44.964000  44.964000  44.723714  44.723714  44.723714  44.723714  44.723714  44.723714  44.723714  44.723714  44.6265  44.6265  44.6265  44.6265  44.6265  44.6265  44.6265  44.6265  44.555556  44.555556  44.555556  44.555556  44.555556  44.555556  44.555556  44.555556
10
2020-01-06 00:00:00+00:00  47.138  47.138  47.138  47.138  47.138  47.138  47.138  47.138  46.456000  46.456000  46.456000  46.456000  46.456000  46.456000  46.456000  46.456000  45.7310  45.7310  45.7310  45.7310  45.7310  45.7310  45.7310  45.7310  45.3376  45.3376  45.3376  45.3376  45.3376  45.3376  45.3376  45.3376  45.112667  45.112667  45.112667  45.112667  45.112667  45.112667  45.112667  45.112667  44.871143  44.871143  44.871143  44.871143  44.871143  44.871143  44.871143  44.871143  44.7115  44.7115  44.7115  44.7115  44.7115  44.7115  44.7115  44.7115  44.660222  44.660222  44.660222  44.660222  44.660222  44.660222  44.660222  44.660222
11
dualma.slow_ma.head(3)
12
dualma_fast_window                10                                                                              15                                                                                      20                                                                                25                                                                                 30                                                                                    35                                                                                      40                                                                                   45
13
dualma_slow_multi                1.5      2.0      2.5        3.0        3.5      4.0        4.5      5.0        1.5        2.0        2.5        3.0        3.5        4.0        4.5        5.0        1.5      2.0      2.5        3.0        3.5        4.0        4.5      5.0        1.5      2.0        2.5        3.0        3.5      4.0        4.5       5.0        1.5        2.0        2.5        3.0        3.5        4.0        4.5      5.0        1.5        2.0        2.5        3.0        3.5        4.0        4.5        5.0        1.5        2.0      2.5        3.0        3.5        4.0        4.5       5.0        1.5        2.0        2.5        3.0        3.5        4.0        4.5        5.0
14
date
15
2020-01-02 00:00:00+00:00  45.824667  45.3025  44.9476  44.816667  44.594571  44.5425  44.440222  44.6384  45.180455  44.816667  44.545676  44.440222  44.717692  45.135167  45.513134  46.025200  44.816667  44.5425  44.6384  45.135167  45.697429  46.307750  46.683111  47.0983  44.545676  44.6384  45.235806  46.025200  46.560460  47.0983  47.997679  48.61136  44.440222  45.135167  46.025200  46.683111  47.425238  48.410917  48.769630  48.8484  44.717692  45.697429  46.560460  47.425238  48.496066  48.803714  48.852357  49.430914  45.135167  46.307750  47.0983  48.410917  48.803714  48.892313  49.622778  50.14240  45.513134  46.683111  47.997679  48.769630  48.852357  49.622778  50.162574  50.375822
16
2020-01-03 00:00:00+00:00  46.128667  45.5025  45.1420  44.964000  44.723714  44.6265  44.555556  44.6660  45.373636  44.964000  44.652162  44.555556  44.741538  45.119167  45.485821  45.984267  44.964000  44.6265  44.6660  45.119167  45.666714  46.291125  46.643333  47.0707  44.652162  44.6660  45.229677  45.984267  46.549080  47.0707  47.936429  48.56848  44.555556  45.119167  45.984267  46.643333  47.349905  48.362083  48.758074  48.8320  44.741538  45.666714  46.549080  47.349905  48.460984  48.784357  48.838471  49.366457  45.119167  46.291125  47.0707  48.362083  48.784357  48.878875  49.584500  50.12260  45.485821  46.643333  47.936429  48.758074  48.838471  49.584500  50.141139  50.379778
17
2020-01-06 00:00:00+00:00  46.456000  45.7310  45.3376  45.112667  44.871143  44.7115  44.660222  44.6908  45.562273  45.112667  44.787297  44.660222  44.773846  45.116667  45.474478  45.950800  45.112667  44.7115  44.6908  45.116667  45.641143  46.267875  46.621889  47.0449  44.787297  44.6908  45.232742  45.950800  46.534598  47.0449  47.864554  48.52880  44.660222  45.116667  45.950800  46.621889  47.278952  48.320667  48.743185  48.8232  44.773846  45.641143  46.534598  47.278952  48.406803  48.770500  48.833885  49.298743  45.116667  46.267875  47.0449  48.320667  48.770500  48.860063  49.552222  50.09115  45.474478  46.621889  47.864554  48.743185  48.833885  49.552222  50.122772  50.388044

svg

21,网格参数-信号计算和可视化#

仅可视化第一列

1
dmac_size.shape: (728, 64)
2
dmac_size.iloc[:3,:3]:
3
dualma_fast_window           10
4
dualma_slow_multi           1.5   2.0   2.5
5
date
6
2020-01-02 00:00:00+00:00  True  True  True
7
2020-01-03 00:00:00+00:00  True  True  True
8
2020-01-06 00:00:00+00:00  True  True  True

svg

1
Start                       2020-01-02 00:00:00+00:00
2
End                         2022-12-30 00:00:00+00:00
3
Period                                            728
4
Total                                             295
5
Rate [%]                                    40.521978
6
First Index                 2020-01-02 00:00:00+00:00
7
Last Index                  2022-12-21 00:00:00+00:00
8
Norm Avg Index [-1, 1]                      -0.160021
9
Distance: Min                                     1.0
10
Distance: Max                                    89.0
11
Distance: Mean                                2.44898
12
Distance: Std                                8.855444
13
Total Partitions                                   13
14
Partition Rate [%]                            4.40678
15
Partition Length: Min                             1.0
16
Partition Length: Max                            52.0
17
Partition Length: Mean                      22.692308
18
Partition Length: Std                       16.428556
19
Partition Distance: Min                           7.0
20
Partition Distance: Max                          89.0
21
Partition Distance: Mean                         36.5
22
Partition Distance: Std                     27.750512
23
Name: (10, 1.5), dtype: object

22,行情,信号的滑窗处理#

a,参数设置和效果预览#

1
# 滚动周期参数设置和大致效果可视化
2
start_end_days=ohlcv.shape[0]
3
bar_days= 80         # 训练，验证集时间长度，以此为单位
4
test_bar_num=2      # 训练集时间长度
5
verify_bar_num=1    # 验证集时间长度
6
verify_overlap=0 # 验证集重叠时间长度
7
pre_test_days=0    # 由于测试集一部分时间用于计算指标，导致实际训练时间不足，这个是一定程度补充的days周期
8
# n取值需要满足:确保验证集合收尾相接
9
# => (n-1)*(verify_bar_num-verify_overlap)+(verify_bar_num+test_bar_num)=start_end_days/bar_days
10
# => n=(start_end_days/bar_days-test_bar_num-verify_overlap)/(verify_bar_num-verify_overlap)
11
calc_n=(start_end_days/bar_days-test_bar_num-verify_overlap)/(verify_bar_num-verify_overlap)
12

13

14
split_kwargs = dict(
15
    n=int(calc_n),
16
    window_len=int(bar_days*(test_bar_num+verify_bar_num)+pre_test_days),
17
    set_lens=(int(bar_days*verify_bar_num),),
18
    left_to_right=False
19
)  # 10 windows, each 2 years long, reserve 180 days for test
20
# 合理设置n，最好确保验证集，连续且无重复
21
pf_kwargs = dict(
22
    direction='longonly',  # long and short
23
    freq='d'
24
)
25
print('split_kwargs:',split_kwargs)
26

27
def roll_in_and_out_samples(price, **kwargs):
28
    return price.vbt.rolling_split(**kwargs)
29

30
price=ohlcv['Close']
31
# 验证：单列数据验证，橘黄色验证集连续且无重复
32
roll_in_and_out_samples(price, **split_kwargs, plot=True, trace_names=['in-sample', 'out-sample']).show_svg()

1
split_kwargs: {'n': 7, 'window_len': 240, 'set_lens': (80,), 'left_to_right': False}

svg

b,根据滑窗参数切分行情数据和信号#

1
in_price.shape: (160, 7)
2
out_price.shape: (80, 7)
3

4
in_price.index: RangeIndex(start=0, stop=160, step=1)
5
in_price.columns: Int64Index([0, 1, 2, 3, 4, 5, 6], dtype='int64', name='split_idx')
6

7
in_price[0:3]:
8
split_idx      0      1      2       3       4       5       6
9
0          48.17  59.78  92.59  219.90  146.56  254.11  250.02
10
1          48.04  58.88  90.00  216.30  153.73  277.60  246.50
11
2          48.28  59.13  94.74  225.04  148.99  275.95  246.30
12

13
###############################
14
in_dmac_size.shape: (160, 448)
15
out_dmac_size.shape: (80, 448)
16

17
in_dmac_size.iloc[:5,:5]:
18
split_idx              0
19
dualma_fast_window    10
20
dualma_slow_multi    1.5   2.0   2.5   3.0   3.5
21
0                   True  True  True  True  True
22
1                   True  True  True  True  True
23
2                   True  True  True  True  True
24
3                   True  True  True  True  True
25
4                   True  True  True  True  True

23,滑窗的收益数据计算#

a,持有参数收益#

在此区间，基础标的物表现

1
def simulate_holding(price, **kwargs):
2
    pf = vbt.Portfolio.from_holding(price, **kwargs)
3
    return pf.sharpe_ratio()
4

5
in_hold_sharpe = simulate_holding(in_price, **pf_kwargs)
6
print(in_hold_sharpe.head(5))
7

8
out_hold_sharpe = simulate_holding(out_price, **pf_kwargs)
9
print(out_hold_sharpe.head(5))

1
split_idx
2
0    2.315678
3
1    3.890261
4
2    1.812302
5
3    1.122310
6
4    2.388496
7
Name: sharpe_ratio, dtype: float64
8
split_idx
9
0    4.885519
10
1   -0.547754
11
2    4.538256
12
3   -0.039085
13
4   -0.527252
14
Name: sharpe_ratio, dtype: float64

b,网格参数收益(训练集和验证集)#

1
in_sharpe.shape: (1792,)
2
split_idx  dualma_fast_window  dualma_slow_multi  sl_stop
3
0          10                  1.5                0.05       1.850726
4
                                                  0.10       1.473377
5
                                                  0.15       1.272865
6
                                                  0.20       1.397542
7
                               2.0                0.05       2.399222
8
                                                               ...
9
6          45                  4.5                0.20      -1.054460
10
                               5.0                0.05      -0.331869
11
                                                  0.10      -1.664299
12
                                                  0.15      -1.487590
13
                                                  0.20      -1.513999
14
Name: sharpe_ratio, Length: 1792, dtype: float64
15

16
split_idx               0                                                                                                                                                                                                                                                                                                                                                                                                                                                               1                                                                                                                                                                                                                                                                                                                                                                                                                                                               2                                                                        \
17
dualma_fast_window     10                                                      15                                                      20                                                      25                                                      30                                                      35                                                      40                                                      45                                                      10                                                      15                                                      20                                                      25                                                      30                                                      35                                                      40                                                      45                                                      10                                                      15
18
dualma_slow_multi     1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5
19
0                   False  False  False  False  False  False  False  False  False  False  False  False   True   True   True   True  False  False   True   True   True   True   True   True  False   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True  False  False  False  False  False  False  False  False  False  False  False
20
1                    True  False  False  False  False  False  False   True  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False
21
2                   False   True   True   True   True  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False
22

23
split_idx                                                                                                                                                                                                                                                                                                                                                                                                  3                                                                                                                                                                                                                                                                                                                                                                                                                                                               4                                                                                                                                                     \
24
dualma_fast_window                                        20                                                      25                                                      30                                                      35                                                      40                                                      45                                                      10                                                      15                                                      20                                                      25                                                      30                                                      35                                                      40                                                      45                                                      10                                                      15                                                      20
25
dualma_slow_multi     3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0
26
0                   False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False   True   True  False  False  False  False  False   True   True   True  False  False   True   True   True   True   True   True  False   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False
27
1                   False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False
28
2                   False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False
29

30
split_idx                                                                                                                                                                                                                                                                                                                     5                                                                                                                                                                                                                                                                                                                                                                                                                                                               6                                                                                                                                                                                                                                  \
31
dualma_fast_window                   25                                                      30                                                      35                                                      40                                                      45                                                      10                                                      15                                                      20                                                      25                                                      30                                                      35                                                      40                                                      45                                                      10                                                      15                                                      20                                                      25                                                      30
32
dualma_slow_multi     4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5
33
0                   False  False  False  False  False   True   True   True   True   True  False  False   True   True   True   True   True   True  False   True   True   True   True   True   True   True  False   True   True   True   True   True   True   True   True   True   True   True   True   True   True   True  False  False  False   True  False   True   True   True  False   True   True   True   True   True   True  False   True   True   True   True  False  False  False  False   True   True   True  False  False  False  False  False   True   True  False  False  False  False  False  False   True  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False   True   True  False  False  False  False   True   True   True   True  False
34
1                   False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False   True   True   True  False   True  False  False  False   True  False  False  False  False  False  False  False  False  False  False  False   True  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False   True  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False
35
2                   False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False   True  False  False  False  False  False   True  False  False  False  False  False   True  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False
36

37
split_idx
38
dualma_fast_window                                                      35                                                      40                                                      45
39
dualma_slow_multi     2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0    1.5    2.0    2.5    3.0    3.5    4.0    4.5    5.0
40
0                   False  False   True   True   True   True   True  False  False   True   True   True   True   True   True  False  False   True   True   True   True   True   True  False   True   True   True   True   True   True   True
41
1                   False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False
42
2                   False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False  False
43
out_sharpe.shape: (1792,)

c,训练集上的最佳参数用于验证集#

大致思路：
01,获取各split_idx的最佳收益(sharp_radio)的参数组合idxmax,也就是fast_window,slow_window,split_idx，三维索引元组
02,按照split_idx进行聚类，取得各split_idx对应的最佳参数。实际含义就是各滑动窗口的最佳参数

v1<简单最大值优选法>
选取，测试集合的最优参数作为验证集参数，如果sharp_ratio就最大，回撤就最小类似这样的简单优选策略。

1
def get_best_index(performance, higher_better=True):
2
    if higher_better:
3
        return performance[performance.groupby('split_idx').idxmax()].index
4
    return performance[performance.groupby('split_idx').idxmin()].index
5
in_test_best_index_basic = get_best_index(in_sharpe)
6

7
merged_df = pd.concat([in_sharpe, in_return,out_sharpe,out_return], axis=1, keys=['in_sharpe', 'in_return','out_sharpe', 'out_return'])
8
print('merged_df[in_test_best_index_basic]')
9
print(merged_df.loc[in_test_best_index_basic])
10

11
# 绘图：参数走势图
12
df_plot_tmp = in_test_best_index_basic.to_frame(index=False)
13
# 将split_idx设置为行索引，并按照split_idx从小到大排序
14
df_plot_tmp.set_index('split_idx', inplace=True)
15
df_plot_tmp.sort_index(inplace=True)
16
df_plot_tmp['dualma_slow_window'] = df_plot_tmp['dualma_fast_window']*df_plot_tmp['dualma_slow_multi']
17
df_plot_tmp[['dualma_fast_window','dualma_slow_window']].vbt.plot().show_svg()

1
merged_df[in_test_best_index_basic]
2
                                                        in_sharpe  in_return  out_sharpe  out_return
3
split_idx dualma_fast_window dualma_slow_multi sl_stop
4
0         40                 1.5               0.05      2.713139   0.576378    0.204789    0.004091
5
1         20                 2.0               0.10      4.260810   1.625152    2.600636    0.278124
6
2         20                 1.5               0.10      3.841407   1.434804    2.621509    0.381941
7
3         10                 3.5               0.10      2.015480   0.526104    0.451947    0.023056
8
4         40                 5.0               0.05      2.835772   0.428931   -1.923102   -0.062478
9
5         10                 3.5               0.05      1.667313   0.159633    3.346740    0.295341
10
6         10                 2.0               0.10      2.966911   0.445291   -3.783652   -0.141299

svg

v2<邻近域优选法>
有些情况下，测试集得到参数会突然发生较大变化，这可能偶发事件导致的，
比如：之前的双均线最佳参数一直是，（20，40），本期突然变成（80，160），显然不大合理，为了避免这种突变，让参数的变化也具有一定连贯性（当然，增加连贯性也一定程度降低过拟合风险）

1
in_test_best_index_nb_coord[:5]
2
MultiIndex([(0, 40, 1.5, 0.05),
3
            (1, 30, 1.5,  0.1),
4
            (2, 20, 1.5,  0.1),
5
            (3, 15, 2.5,  0.1),
6
            (4, 10, 3.5, 0.05)],
7
           names=['split_idx', 'dualma_fast_window', 'dualma_slow_multi', 'sl_stop'])

svg

v3<邻居权重优选法-均值>
在评估一组参数是否最佳时，并不单纯观察此参数本身是否最优，而是综合考虑参数本以及参数的邻居表现。
比如： 0.5 0.7 0.5 0.2 0.2
0.8 0.7 0.6 0.9 0.2
0.5 0.7 0.5 0.2 0.2
按照基础的最大值法，则选择0.9，但是0.9的邻居表现均不佳。
定义：新取值=原值 + (邻居的平均值)
则可以综合考虑参数本身和参数邻居点的表现。

1
in_test_best_index_nb_mean[:5]
2
MultiIndex([(0, 25, 1.5, 0.05),
3
            (1, 20, 2.0, 0.15),
4
            (2, 20, 1.5, 0.15),
5
            (3, 10, 3.5, 0.05),
6
            (4, 45, 5.0, 0.05)],
7
           names=['split_idx', 'dualma_fast_window', 'dualma_slow_multi', 'sl_stop'])

svg

v4<邻居权重优选法-中位数>
由于均值受极值影响较大，可以考虑用 median( 多个邻居)，代替上面”邻居的平均值”。

1
in_test_best_index_nb_median[:5]
2
MultiIndex([(0, 30, 2.0, 0.05),
3
            (1, 20, 2.0,  0.1),
4
            (2, 20, 1.5,  0.1),
5
            (3, 10, 3.5, 0.05),
6
            (4, 45, 5.0, 0.05)],
7
           names=['split_idx', 'dualma_fast_window', 'dualma_slow_multi', 'sl_stop'])

svg

将滚动获取的最佳参数用于验证集，统计收益信息

24,sharp ratio的汇总可视化#

basic为例的基础分析视图

1
cv_results_df = pd.DataFrame({
2
    'in_sample_hold': in_hold_sharpe.values,
3
    'in_sample_median': in_sharpe.groupby('split_idx').median().values,
4
    'in_sample_best': in_test_best_sharpe_basic.values,
5
    'out_sample_hold': out_hold_sharpe.values,
6
    'out_sample_median': out_sharpe.groupby('split_idx').median().values,
7
    'out_sample_test': out_test_sharpe_basic.values
8
})
9

10
color_schema = vbt.settings['plotting']['color_schema']
11

12
cv_results_df.vbt.plot(
13
    trace_kwargs=[
14
        dict(line_color=color_schema['blue']),
15
        dict(line_color=color_schema['blue'], line_dash='dash'),
16
        dict(line_color=color_schema['blue'], line_dash='dot'),
17
        dict(line_color=color_schema['orange']),
18
        dict(line_color=color_schema['orange'], line_dash='dash'),
19
        dict(line_color=color_schema['orange'], line_dash='dot')
20
    ]
21
).show_svg()

svg

关注点：

蓝色部分正常排序是(从上到下)：点线，实现，线段，

橘色部分

实线对实线
说明测试集和验证集的周期收益情况，二者同时出现0轴同侧较好（同时上涨，同时下跌，保持行情的稳定性or延续性）

线段对线段
二者一方面随着各自颜色的实线趋势变化（受各自实线影响较大），其他应该无必然联系

点线对点线
蓝色点高于橘色点线，蓝色是训练集内最佳，橘色则是训练集得到最优参数用于验证集结果收益，大概率低于验证集。

测试，验证集时间长度差异，引入偏差
由于测试集一般是验证集的2-3倍（或更多），对于单边行情(假如上涨)，则(测试集的)实线收益。蓝色线大概率位于橘色线上方。
如果下跌，则相反。蓝色由于时间长，大概率位于橘色下方。

注意： 01，202406，对于当前case，y周取值为sharp ratio夏普比，而非收益率。所以数据点高低并不反映收益率。所以，以上结论需要稍斟酌，并不完全准确。

4种优选方法的训练集夏普sharp ratio

1
cv_results_df = pd.DataFrame({
2
    'in_sample_hold': in_hold_sharpe.values,
3
    'in_sample_best_basic': in_sharpe[in_test_best_index_basic].values,
4
    'in_sample_best_coord': in_sharpe[in_test_best_index_nb_coord].values,
5
    'in_sample_best_mean': in_sharpe[in_test_best_index_nb_mean].values,
6
    'in_sample_best_median': in_sharpe[in_test_best_index_nb_median].values,
7
})
8

9

10
color_schema = vbt.settings['plotting']['color_schema']
11

12
cv_results_df.vbt.plot(
13
    trace_kwargs=[
14
        dict(line_color=color_schema['blue']),
15
        dict(line_color=color_schema['green']),
16
        dict(line_color=color_schema['red']),
17
        dict(line_color=color_schema['cyan']),
18
        dict(line_color=color_schema['orange'])
19
    ]
20
).show_svg()

svg

4种优选方法的验证集夏普sharp ratio

1
cv_results_df = pd.DataFrame({
2
    'out_sample_hold': out_hold_sharpe.values,
3
    'out_sample_test_basic': out_test_sharpe_basic.values,
4
    'out_sample_test_coord': out_test_sharpe_coord.values,
5
    'out_sample_test_mean': out_test_sharpe_mean.values,
6
    'out_sample_test_median': out_test_sharpe_median.values
7
})
8

9
color_schema = vbt.settings['plotting']['color_schema']
10

11
cv_results_df.vbt.plot(
12
    trace_kwargs=[
13
         dict(line_color=color_schema['blue']),
14
        dict(line_color=color_schema['green']),
15
        dict(line_color=color_schema['red']),
16
        dict(line_color=color_schema['cyan']),
17
        dict(line_color=color_schema['orange'])
18
    ]
19
).show_svg()

svg

25,滚动回测收益可视化#

1
# 测试集：原始价格变动
2
in_price_org=in_price.iloc[-1, :]/in_price.iloc[0, :]
3
print('in_price_org shape:',in_price_org.shape)
4
print('in_price_org.head(5)')
5
print(in_price_org.head(5))
6

7

8
cv_results_df = pd.DataFrame({
9
    'out_price_org':  in_price_org.cumprod(),
10
    'in_test_best_return_basic': (in_test_best_return_basic.values+1).cumprod(),
11
    'in_test_best_return_coord': (in_test_best_return_nb_coord.values+1).cumprod(),
12
    'in_test_best_return_mean': (in_test_best_return_nb_mean.values+1).cumprod(),
13
    'in_test_best_return_median': (in_test_best_return_nb_median.values+1).cumprod(),
14

15
})
16

17
color_dmac_pfschema = vbt.settings['plotting']['color_schema']
18

19

20
cv_results_df.vbt.plot(
21
    trace_kwargs=[
22
         dict(line_color=color_schema['blue']),
23
        dict(line_color=color_schema['green']),
24
        dict(line_color=color_schema['red']),
25
        dict(line_color=color_schema['cyan']),
26
        dict(line_color=color_schema['orange'])
27
    ]
28
).show_svg()
29

30

31

32
# 验证集：原始价格变动
33
out_price_org=out_price.iloc[-1, :]/out_price.iloc[0, :]
34
print('out_price_org shape:',out_price_org.shape)
35
print('out_price_org.head(5)')
36
print(out_price_org.head(5))
37

38
print()
39
print('out_test_return_basic shape:',out_test_return_basic.shape)
40
print('out_test_return_basic.head(5) + 1')
41
print(out_test_return_basic.head(5)+1)
42

43
cv_results_df = pd.DataFrame({
44
    'out_price_org':  out_price_org.cumprod(),
45
    'out_test_return_basic': (out_test_return_basic.values+1).cumprod(),
46
    'out_test_return_coord': (out_test_return_coord.values+1).cumprod(),
47
    'out_test_return_mean': (out_test_return_mean.values+1).cumprod(),
48
    'out_test_return_median': (out_test_return_median.values+1).cumprod(),
49
})
50

51
color_dmac_pfschema = vbt.settings['plotting']['color_schema']
52

53

54
cv_results_df.vbt.plot(
55
    trace_kwargs=[
56
        dict(line_color=color_schema['blue']),
57
        dict(line_color=color_schema['green']),
58
        dict(line_color=color_schema['red']),
59
        dict(line_color=color_schema['cyan']),
60
        dict(line_color=color_schema['orange'])
61
    ]
62
).show_svg()

1
in_price_org shape: (7,)
2
in_price_org.head(5)
3
split_idx
4
0    1.772680
5
1    2.987621
6
2    1.620045
7
3    1.282265
8
4    1.822666
9
dtype: float64

svg

1
out_price_org shape: (7,)
2
out_price_org.head(5)
3
split_idx
4
0    2.210941
5
1    0.876075
6
2    2.001737
7
3    0.971119
8
4    0.902879
9
dtype: float64
10

11
out_test_return_basic shape: (7,)
12
out_test_return_basic.head(5) + 1
13
split_idx  dualma_fast_window  dualma_slow_multi  sl_stop
14
0          40                  1.5                0.05       1.004091
15
1          20                  2.0                0.10       1.278124
16
2          20                  1.5                0.10       1.381941
17
3          10                  3.5                0.10       1.023056
18
4          40                  5.0                0.05       0.937522
19
Name: total_return, dtype: float64

svg

上图可见，以上参数优选方法表现基本接近（也符合之前的sharp ratio接近的特征）

26,计算正确性验证(略)#

27,回测结果汇总#

std_indicator#

过滤器规则：

1
std_close_wbuf = ohlcv_wbuf['Close'].rolling(window=20).std()
2
std_close_ma_wbuf = std_close_wbuf.rolling(window=20).mean()
3
std_close=std_close_wbuf[wobuf_mask]
4
std_close_ma=std_close_ma_wbuf[wobuf_mask]
5
std_indicator = (std_close > std_close_ma )

4种优选方法的训练集夏普sharp ratio
svg

4种优选方法的验证集夏普sharp ratio
svg 样本内滚动收益
svg

样本外滚动收益 svg

diff_indicator#