阿拉丁和灯的知识库
Page tree
ConfigureSpace tools
Skip to end of metadata
Go to start of metadata

 

1. 

 

 

Transformer模型的代码也在里面了

 

使用方法

下一步计划

1. 用MNLI或者MRPC的数据，跑通fine tuning过程

2. 准备工单数据，参考MNLI或者MRPC。（我们的任务接近MRPC，完全可以参考它同样的方法）

MNLI跟MRPC没有本质区别，如下图：

XNLI和Mrpc Processor的区别

试验运行

1. 安装python

2. 升级Pip

/d/python36/python.exe -m pip install --upgrade pip3

3. 安装tensorflow

pip3 install tensorflow --ignore-installed

4. 准备代码和数据

4.1 下载BERT模型（uncasedxxx, chinesexxx），解压缩到bert-model目录

4.2 下载bert代码，解压缩到bert目录

4.3 下载MRPC数据集，解压数据文件并分别改名为dev.tsv, train.tsv, test.tsv

4.3 编辑运行命令行

运行MRPC的命令行(cygwin下)

#export BERT_BASE_DIR=/D/Dev/MyProjects/IcbuTickets/bert-model/chinese_L-12_H-768_A-12
export BERT_BASE_DIR=D:\\Dev\\MyProjects\\IcbuTickets\\bert-model\\uncased_L-12_H-768_A-12
export DATA_DIR=D:\\Dev\\MyProjects\\IcbuTickets\\mrpc

/d/python36/python.exe ../../bert/run_classifier.py \
--task_name=MRPC \
--do_train=true \
--do_eval=true \
--data_dir=$DATA_DIR \
--vocab_file=$BERT_BASE_DIR/vocab.txt \
--bert_config_file=$BERT_BASE_DIR/bert_config.json \
--init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \
--max_seq_length=128 \
--train_batch_size=32 \
--learning_rate=5e-5 \
--num_train_epochs=0.1 \
--output_dir=D:\\Dev\\MyProjects\\mrpc_output

注意：直接参数的路径按uninx，传递给python代码的参数路径按Window格式

参考：https://www.tutorialkart.com/deep-learning/install-tensorflow-for-gpu/

https://medium.com/@shaolinkhoa/install-tensorflow-gpu-2-0-alpha-on-anaconda-for-windows-10-ubuntu-ced099010b21

https://github.com/dragen1860/TensorFlow-2.x-Tutorials/tree/master/01-TF2.0-Overview

https://www.tensorflow.org/tutorials/quickstart/beginner

完整过程

1. 安装python 3.6和pip3

https://www.rosehosting.com/blog/how-to-install-python-3-6-4-on-centos-7/

命令行

 

[root@iZ8vb8akuodqsoo0dl0512Z ~]# history
1 python3.6 -V
2 pip --version
3 pip3 --version
4 python3.6 -m pip install --upgrade pip3
5 pip install --upgrade pip
6 pip3
7 python3.6 -m pip install --upgrade pip
8 pip
9 pip3
10 pip3 install tensorflow --ignore-installed
11 python3.6 -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
12 history

安装

  说明
 The following NVIDIA® software must be installed on your systemhttps://www.tensorflow.org/install/gpu
1云主机申请时勾选自动安装
2下载安装
3
  • CUPTI ships with the CUDA Toolkit.
 
4 
5
  • (Optional) NCCL 2.2 for multiple GPU support.
  • (Optional) TensorRT 4.0 to improve latency and throughput for inference on some models.
 

step2


Docker方式

https://www.tensorflow.org/install/docker

1. 安装Docker CE

https://docs.docker.com/install/linux/docker-ce/centos/#set-up-the-repository

 
sudo yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-selinux docker-engine-selinux docker-engine
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install docker-ce

#注意这里要reboot

sudo systemctl start docker

 

2. 安装Nvidia-docker

https://github.com/NVIDIA/nvidia-docker

用Docker-CE版

 Click here to expand...

 

[root@iZ8vbfsflb71ljy03o5gxhZ ~]# docker run --runtime=nvidia -it --rm tensorflow/tensorflow:latest-gpu python3 --version
Python 3.5.2

 

Volume
docker volume create vol-icbu-tickets
docker volume inspect vol-icbu-tickets

docker run 命令中间增加

--mount source=vol-icbu-tickets,target=/icbu-tickets

参考：https://docs.docker.com/storage/volumes/#create-and-manage-volumes

启动docker tensorflow容器

 
docker run --runtime=nvidia --mount source=vol-icbu-tickets,target=/icbu-tickets -it --rm tensorflow/tensorflow:latest-gpu bash
 
python /icbu-tickets/bert/run_classifier.py --task_name=icbu-tickets --do_train=true --do_eval=true --data_dir=/icbu-tickets/icbu-tickets --vocab_file=/icbu-tickets/bert-model/chinese_L-12_H-768_A-12/vocab.txt --bert_config_file=/icbu-tickets/bert-model/chinese_L-12_H-768_A-12/bert_config.json --init_checkpoint=/icbu-tickets/bert-model/chinese_L-12_H-768_A-12/bert_model.ckpt --max_seq_length=128 --train_batch_size=32 --learning_rate=5e-5 --num_train_epochs=0.1 --output_dir=/icbu-tickets/output

 

 

 

问题解决

https://stackoverflow.com/questions/42754779/docker-runc-not-installed-on-system

docker-runc找不到的问题
In reference to the top answer, introduce the sym link in /usr/bin to enable it in global path
sudo ln -s /usr/libexec/docker/docker-runc-current /usr/bin/docker-runc
编码问题 - 读取label时的编码问题

ICBU/av\u8ba4\u8bc1

tokenization.convert_to_unicode()

编码问题2-读取数据时的编码问题

https://stackoverflow.com/questions/21129020/how-to-fix-unicodedecodeerror-ascii-codec-cant-decode-byte

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 5: ordinal not in range(128)

import sys 
reload(sys) 
sys.setdefaultencoding('utf8')

module找不到的问题

No module named html2text

进到容器里面安装

 

 

out of memory的问题

从Docker方面尝试解决

https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details

从TensorFlow方面尝试解决

https://github.com/google-research/bert#out-of-memory-issues

 

  • No labels