I am trying to fit a SVM model where my predicted true values are multiindexes that match. The problem is that I dont know how to specify that the multiindixes are the true values.
I cannot use the record linkage classification step because it is not very flexible.
from sklearn.svm import SVC
golden_pairs = filter_tests_new_df[:training_value]
golden_matches_index = golden_pairs[golden_pairs['ev_2'] == 1].index
# This is a multiindex type
svm = SVC(gamma='auto')
svm.fit(golden_pairs, golden_matches_index)
# I dont know how to specify that the golden_matches_index are the good matches
# Predict the match status for all record pairs
result_svm = svm.predict(test_pairs[columns_to_keep])
You don't have to specify the index
, instead use the generated boolean Series
as the labels for the classification.
Here's an example.
# Sample data
data = pd.DataFrame({'a': [1, 2, 3],
'b': [1, 1, 0]})
data
a b
0 1 1
1 2 1
2 3 0
# Generating labels
data['b'] == 1
0 True
1 True
2 False
Name: b, dtype: bool
# Can convert them to integer if required
(data['b'] == 1).astype(int)
0 1
1 1
2 0
Name: b, dtype: int64
Based on your code, I think this should do the trick
# Boolean
golden_pairs['ev_2'] == 1
# Integer
(golden_pairs['ev_2'] == 1).astype(int)