CNN句子分类(Convolutional Neural Networks for Sentence Classification)

2017-08-05

Convolutional Neural Networks for Sentence Classification

遇到的最严重问题，优先列出：

新版本theano.tensor.signal包下已经不包含downsample模块，然后利用theano.tensor.signal.pool.pool_2d()方法代替了theano.tensor.signal.downsample.max_pool_2d()方法

实验

1.实验所需的是theano框架

安装稳定的最前沿版本：

1	pip install --user --no-deps git+https://github.com/Theano/Theano.git#egg=Theano

2.gpuarray 的安装：为了利用gpu加速运行

参照官网gpuarray网址

1）下载安装包

1 2	git clone https://github.com/Theano/libgpuarray.git cd libgpuarray

2）安装

For libgpuarray

cd <dir> # 这一步如果紧接着上方可以省去，主要是为了进入你下载的目录
mkdir Build
cd Build
# you can pass -DCMAKE_INSTALL_PREFIX=/path/to/somewhere to install to an alternate location
cmake .. -DCMAKE_BUILD_TYPE=Release # or Debug if you are investigating a crash # 这一步我的mac上通过brew install cmake安装cmake后执行
make
make install
cd ..

For pygpu:

1
2
3

# This must be done after libgpuarray is installed as per instructions above.
python setup.py build
python setup.py install

If you installed libgpuarray in a path that isn’t a default one, you will need to specify where it is. Replace the first line by something like this:

1	python setup.py build_ext -L $MY_PREFIX/lib -I $MY_PREFIX/include

If installed globally under Linux (in /usr/local), you might have to run:

1	$ sudo ldconfig

虽然没爱搞懂，但最后一步我没有执行

github源代码

实验数据:在这个页面我下载的是GoogleNews-vectors-negative300.bin.gz

这个压缩包解压缩后就是已经用word2vec预处理好的数据

然后就是根据这个例子做实验

Data Preprocessing

To process the raw data, run

1	python process_data.py path

where path points to the word2vec binary file (i.e. GoogleNews-vectors-negative300.bin file). This will create a pickle object called mr.p in the same folder, which contains the dataset in the right format.

Note: This will create the dataset with different fold-assignments than was used in the paper. You should still be getting a CV score of >81% with CNN-nonstatic model, though.

Running the models (CPU)

这一步会遇到一个很严重的问题就是前面说的新版本theano.tensor.signal包下已经不包含downsample模块，然后利用theano.tensor.signal.pool.pool_2d()方法代替了theano.tensor.signal.downsample.max_pool_2d()方法，所以需要修改conv_net_classes.py中相关的地方。

Example commands:

1
2
3

THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 python conv_net_sentence.py -nonstatic -rand
THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 python conv_net_sentence.py -static -word2vec
THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 python conv_net_sentence.py -nonstatic -word2vec

This will run the CNN-rand, CNN-static, and CNN-nonstatic models respectively in the paper.

Using the GPU

GPU will result in a good 10x to 20x speed-up, so it is highly recommended. To use the GPU, simply change device=cpu to device=gpu (or whichever gpu you are using). For example:

1	THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python conv_net_sentence.py -nonstatic -word2vec

Example output

CPU output:

1
2
3

epoch: 1, training time: 219.72 secs, train perf: 81.79 %, val perf: 79.26 %
epoch: 2, training time: 219.55 secs, train perf: 82.64 %, val perf: 76.84 %
epoch: 3, training time: 219.54 secs, train perf: 92.06 %, val perf: 80.95 %

GPU output:

1
2
3

epoch: 1, training time: 16.49 secs, train perf: 81.80 %, val perf: 78.32 %
epoch: 2, training time: 16.12 secs, train perf: 82.53 %, val perf: 76.74 %
epoch: 3, training time: 16.16 secs, train perf: 91.87 %, val perf: 81.37 %

###