This model is directly analagous to this Tesnsorflow's LM. In fact, the reader is directly taken from its older version
See this blogpost.
python ptb-lm.py --data=[PATH_TO_DATA]
Default params should result in Test perplexity of ~78.04. Your actual result will vary due to random initialization. This basically matches results from TF's tutorial, only faster.
On GTX 1080 I am getting around 7,400 wps.
- lm.py - language model description
- reader.py - slightly older version of TF's PTB reader which yields numpy arrays as batches
- ptb-lm.py - driver script
- Python 3 (I used Anaconda distribution)
- PyTorch (I used 0.1.12)