artificial-intelligencetranslationopennmtnmt

Marian nmt : marian-decoder, empty translation


I am interested in running marian-decoder (cli tool) for an english-french translation. Linux. Running Marian 1.12.

Apparently, what I currently do, does not yield any result. Let me show you the steps and please tell me where I am wrong.

1: compile marian-nmt, following the documentation:

cmake -DUSE_SENTENCEPIECE=on -DCOMPILE_CPU=on -DCOMPILE_CUDA=off ..
make -j24

Everything goes fine.

2: Download an opus file: https://object.pouta.csc.fi/OPUS-MT-models/en-fr/opus-2020-02-26.zip

3: Adjusting the directory via cd. Run

../build/marian-decoder -m opus-2020-02-26_en_fr/opus.spm32k-spm32k.transformer-align.model1.npz.best-perplexity.npz -v opus-2020-02-26_en_fr/source.spm opus-2020-02-26_en_fr/target.spm --cpu-threads 1 -i test.txt -o out.txt

alternatively

../build/marian-decoder -m opus-2020-02-26_en_fr/opus.spm32k-spm32k.transformer-align.model1.npz.best-perplexity.npz -v opus-2020-02-26_en_fr/opus.spm32k-spm32k.vocab.yml opus-2020-02-26_en_fr/opus.spm32k-spm32k.vocab.yml --cpu-threads 1 -i test.txt -o out.txt

No error or anything strange in the log.

4: input file has sentence: "Hi, how are you?".

Issue:

The out.txt file remains empty. What did I miss in the process ?

Thanks


Solution

  • So apparently, I had to spm_encode my input text first, then run the above commands.