How can I process Persian texts using Rapid Miner?
I am working on a persian classification project. Persian texts is very similar to arabic texts. when I use Tokenize, it does not show any word in its wordlist page and in Example Set Page, The Image below will be shown:
I need to classify persian text to some category, but I dont know How?.
I Follow some steps like this:
1- Read Excel(using Read Excel component) dataset with 2 column => col1:persian Text ,col2: Category
2- I use Set role component to labeling data
3- I use Process Documents from Data component containing :(Tokenize(with any mode not change anythings) and Filter Token(min:5,max:25) inside it)
4- Then I use Cross Validation Component to train with SVM or Basian and in test mode to get performance.
The program runs correctly and performance is not bad for e.g accuracy is 50% but I think my work is Wrong.
Any help would be appreciated.
first, make sure your text data have UTF-8 encoding and if u use filter tokens(by length) 5 is too much for minimum try 2 or at least 3 also, I recommend using Filter Stopwords (Dictionary) operator and the dictionary should have Persian stopwords in each line hope it will help u
- RuntimeError: weight tensor should be defined either for all 1000 classes or no classes but got weight tensor of shape: [5]
- Scikit train_test_split by an index
- Matlab cross validation and K-NN
- Choosing k for KNN in Matlab
- How to process files in FastAPI from multiple clients without saving the files to disk
- Plot a decision tree from HistGratientBoostingClassifier
- Sequence to Sequence classification with CNN-LSTM model in keras
- Finding optimal cut-off points for dividing a variable into equal groups considering ties in which quantiles do not work
- XGBoost XGBClassifier Defaults in Python
- Use scikit-learn to classify into multiple categories
- Machine Learning: Image classification into 3 classes (Dog or Cat or Neither) using Convolutional NN
- How to import dataset for image sequence classification?
- How to use Machine Learning to find the pattern customer profile?
- rpart() decision tree fails to generate splits (decision tree with only one node (the root node))
- Difference between Generative, Discriminating and Parametric, Nonparametric Algorithm/Model
- Generating documents from LDA topic model
- How can I know training data is enough for machine learning
- I am struggliing spilting the classifers catagory and displaying then in 2 rows. I am using python and tkinter to build this GUI
- Set parameters for classifier and use it without fitting
- Resolve ValueError more than 1 class in BinaryRelevance package
- ROC curve *without* a model estimator?
- ValueError: The number of classes has to be greater than one (python)
- Mixing categorial and continuous data in Naive Bayes classifier using scikit-learn
- Passing an array to numpy.dot() in Python implementation of Perceptron Learning Model
- NameError: name 'plot_confusion_matrix' is not defined
- Why do the sensitivity (recall) values differ between classification_report and precision_recall_fscore_support in a loop?
- How to compute sens slope with yearly data frame with value lower than 3?
- How to plot SHAP summary plots for all classes in multiclass classification
- Scikit-learn: How to obtain True Positive, True Negative, False Positive and False Negative
- Loss remains unchanged in a multiclass classification deep model