Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

How can I do Train And Test step in Giza++?

Tags:

machine-learning

nlp

giza++

In artificial intelligence methods we have two stages of training. These stages are data and testing.

In the training stage we give a huge amount of data to a system and we normally test it with smaller volume of data. Then we evaluate the output.

Now the question is can this training be done through the built in functionality embedded in GIZA++ or we should write a separate application for that?

If we should write a separate application can anybody help me by suggesting an already written application? Or a manual? Note: I want to have an alignment program not a statistical machine translation

I would prefer to train in Giza++ so I can test with unobserved data.

Thanks in advance.

like image 973
m-Abrontan Avatar asked Aug 29 '12 08:08

m-Abrontan

People also ask

Which method is used to generate training and testing data?

Train/Test is a method to measure the accuracy of your model. It is called Train/Test because you split the the data set into two sets: a training set and a testing set. 80% for training, and 20% for testing. You train the model using the training set.

What is meant by training set and test set?

training set—a subset to train a model. test set—a subset to test the trained model.

Why do we train and test data?

By using similar data for training and testing, you can minimize the effects of data discrepancies and better understand the characteristics of the model. After a model has been processed by using the training set, you test the model by making predictions against the test set.

What is the difference between train and test data?

The main difference between training data and testing data is that training data is the subset of original data that is used to train the machine learning model, whereas testing data is used to check the accuracy of the model. The training dataset is generally larger in size compared to the testing dataset.

1 Answers

This should be a good starting point to train a baseline mt system using Moses.
Normally GIZA++ is used for word-aligning your parallel corpus.
You'll need some other tools such as IRSTLM, SRILM, or KenLM for language model estimation.

like image 114
Mortaza Doulaty Avatar answered Oct 19 '22 19:10

Mortaza Doulaty
Sign in to Comment

Related questions

Why does sklearn Imputer need to fit?
Tensor is not an element of this graph
What's the difference between LSTM() and LSTMCell()?
Is there a better way to guess possible unknown variables without brute force than I am doing? Machine learning? [duplicate]
What is the meaning of the nu parameter in Scikit-Learn's SVM class?
keras BatchNormalization axis clarification
How to disable dropout while prediction in keras?
ValueError: Variable rnn/basic_rnn_cell/kernel already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope?
Clustering Algorithm for Mapping Application
Batch normalization instead of input normalization
Tensorflow mean squared error loss function
How does mask_zero in Keras Embedding layer work?
Unit Testing Machine Learning Code
What is OOF approach in machine learning?
Difference between Dense and Activation layer in Keras
Show training and validation accuracy in TensorFlow using same graph
Difference between cross_val_score and cross_val_predict
Difference between parameters, features and class in Machine Learning
Tensorflow Keras Copy Weights From One Model to Another
Why the cost function of logistic regression has a logarithmic expression?

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!