Page tree
Skip to end of metadata
Go to start of metadata

 

1. 

 

 

Transformer模型的代码也在里面了

 

使用方法

下一步计划

1. 用MNLI或者MRPC的数据,跑通fine tuning过程

2. 准备工单数据,参考MNLI或者MRPC。(我们的任务接近MRPC,完全可以参考它同样的方法)

MNLI跟MRPC没有本质区别,如下图:

XNLI和Mrpc Processor的区别

试验运行

1. 安装python

2. 升级Pip

/d/python36/python.exe -m pip install --upgrade pip3

3. 安装tensorflow

pip3 install tensorflow --ignore-installed

4. 准备代码和数据

4.1 下载BERT模型(uncasedxxx, chinesexxx),解压缩到bert-model目录

4.2 下载bert代码,解压缩到bert目录

4.3 下载MRPC数据集,解压数据文件并分别改名为dev.tsv, train.tsv, test.tsv

4.3 编辑运行命令行

运行MRPC的命令行(cygwin下)

#export BERT_BASE_DIR=/D/Dev/MyProjects/IcbuTickets/bert-model/chinese_L-12_H-768_A-12
export BERT_BASE_DIR=D:\\Dev\\MyProjects\\IcbuTickets\\bert-model\\uncased_L-12_H-768_A-12
export DATA_DIR=D:\\Dev\\MyProjects\\IcbuTickets\\mrpc

/d/python36/python.exe ../../bert/run_classifier.py \
--task_name=MRPC \
--do_train=true \
--do_eval=true \
--data_dir=$DATA_DIR \
--vocab_file=$BERT_BASE_DIR/vocab.txt \
--bert_config_file=$BERT_BASE_DIR/bert_config.json \
--init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \
--max_seq_length=128 \
--train_batch_size=32 \
--learning_rate=5e-5 \
--num_train_epochs=0.1 \
--output_dir=D:\\Dev\\MyProjects\\mrpc_output

注意:直接参数的路径按uninx,传递给python代码的参数路径按Window格式

参考:https://www.tutorialkart.com/deep-learning/install-tensorflow-for-gpu/

https://medium.com/@shaolinkhoa/install-tensorflow-gpu-2-0-alpha-on-anaconda-for-windows-10-ubuntu-ced099010b21

https://github.com/dragen1860/TensorFlow-2.x-Tutorials/tree/master/01-TF2.0-Overview

https://www.tensorflow.org/tutorials/quickstart/beginner

完整过程

1. 安装python 3.6和pip3

https://www.rosehosting.com/blog/how-to-install-python-3-6-4-on-centos-7/

命令行

 

[root@iZ8vb8akuodqsoo0dl0512Z ~]# history
1 python3.6 -V
2 pip --version
3 pip3 --version
4 python3.6 -m pip install --upgrade pip3
5 pip install --upgrade pip
6 pip3
7 python3.6 -m pip install --upgrade pip
8 pip
9 pip3
10 pip3 install tensorflow --ignore-installed
11 python3.6 -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
12 history

安装

  说明
 The following NVIDIA® software must be installed on your systemhttps://www.tensorflow.org/install/gpu
1云主机申请时勾选自动安装
2下载安装
3
  • CUPTI ships with the CUDA Toolkit.
 
4 
5
  • (Optional) NCCL 2.2 for multiple GPU support.
  • (Optional) TensorRT 4.0 to improve latency and throughput for inference on some models.
 

step2


Docker方式

https://www.tensorflow.org/install/docker

1. 安装Docker CE

https://docs.docker.com/install/linux/docker-ce/centos/#set-up-the-repository

 
sudo yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-selinux docker-engine-selinux docker-engine
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install docker-ce

#注意这里要reboot

sudo systemctl start docker

 

2. 安装Nvidia-docker

https://github.com/NVIDIA/nvidia-docker

用Docker-CE版

 Click here to expand...

 

[root@iZ8vbfsflb71ljy03o5gxhZ ~]# docker run --runtime=nvidia -it --rm tensorflow/tensorflow:latest-gpu python3 --version
Python 3.5.2

 

Volume
docker volume create vol-icbu-tickets
docker volume inspect vol-icbu-tickets

docker run 命令中间增加

--mount source=vol-icbu-tickets,target=/icbu-tickets

参考:https://docs.docker.com/storage/volumes/#create-and-manage-volumes

启动docker tensorflow容器

 
docker run --runtime=nvidia --mount source=vol-icbu-tickets,target=/icbu-tickets -it --rm tensorflow/tensorflow:latest-gpu bash
 
python /icbu-tickets/bert/run_classifier.py --task_name=icbu-tickets --do_train=true --do_eval=true --data_dir=/icbu-tickets/icbu-tickets --vocab_file=/icbu-tickets/bert-model/chinese_L-12_H-768_A-12/vocab.txt --bert_config_file=/icbu-tickets/bert-model/chinese_L-12_H-768_A-12/bert_config.json --init_checkpoint=/icbu-tickets/bert-model/chinese_L-12_H-768_A-12/bert_model.ckpt --max_seq_length=128 --train_batch_size=32 --learning_rate=5e-5 --num_train_epochs=0.1 --output_dir=/icbu-tickets/output

 

 

 

问题解决

https://stackoverflow.com/questions/42754779/docker-runc-not-installed-on-system

docker-runc找不到的问题
In reference to the top answer, introduce the sym link in /usr/bin to enable it in global path
sudo ln -s /usr/libexec/docker/docker-runc-current /usr/bin/docker-runc
编码问题 - 读取label时的编码问题

ICBU/av\u8ba4\u8bc1

tokenization.convert_to_unicode()

编码问题2-读取数据时的编码问题

https://stackoverflow.com/questions/21129020/how-to-fix-unicodedecodeerror-ascii-codec-cant-decode-byte

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 5: ordinal not in range(128)

import sys 
reload(sys) 
sys.setdefaultencoding('utf8')

module找不到的问题

No module named html2text

进到容器里面安装

 

 

  • No labels