82 lines
4.3 KiB
Plaintext
82 lines
4.3 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a9febc82",
|
|
"metadata": {
|
|
"origin_pos": 0
|
|
},
|
|
"source": [
|
|
"# Natural Language Processing: Applications\n",
|
|
":label:`chap_nlp_app`\n",
|
|
"\n",
|
|
"We have seen how to represent tokens in text sequences and train their representations in :numref:`chap_nlp_pretrain`.\n",
|
|
"Such pretrained text representations can be fed to various models for different downstream natural language processing tasks.\n",
|
|
"\n",
|
|
"In fact,\n",
|
|
"earlier chapters have already discussed some natural language processing applications\n",
|
|
"*without pretraining*,\n",
|
|
"just for explaining deep learning architectures.\n",
|
|
"For instance, in :numref:`chap_rnn`,\n",
|
|
"we have relied on RNNs to design language models to generate novella-like text.\n",
|
|
"In :numref:`chap_modern_rnn` and :numref:`chap_attention-and-transformers`,\n",
|
|
"we have also designed models based on RNNs and attention mechanisms for machine translation.\n",
|
|
"\n",
|
|
"However, this book does not intend to cover all such applications in a comprehensive manner.\n",
|
|
"Instead,\n",
|
|
"our focus is on *how to apply (deep) representation learning of languages to addressing natural language processing problems*.\n",
|
|
"Given pretrained text representations,\n",
|
|
"this chapter will explore two \n",
|
|
"popular and representative\n",
|
|
"downstream natural language processing tasks:\n",
|
|
"sentiment analysis and natural language inference,\n",
|
|
"which analyze single text and relationships of text pairs, respectively.\n",
|
|
"\n",
|
|
"\n",
|
|
":label:`fig_nlp-map-app`\n",
|
|
"\n",
|
|
"As depicted in :numref:`fig_nlp-map-app`,\n",
|
|
"this chapter focuses on describing the basic ideas of designing natural language processing models using different types of deep learning architectures, such as MLPs, CNNs, RNNs, and attention.\n",
|
|
"Though it is possible to combine any pretrained text representations with any architecture for either application in :numref:`fig_nlp-map-app`,\n",
|
|
"we select a few representative combinations.\n",
|
|
"Specifically, we will explore popular architectures based on RNNs and CNNs for sentiment analysis.\n",
|
|
"For natural language inference, we choose attention and MLPs to demonstrate how to analyze text pairs.\n",
|
|
"In the end, we introduce how to fine-tune a pretrained BERT model\n",
|
|
"for a wide range of natural language processing applications,\n",
|
|
"such as on a sequence level (single text classification and text pair classification)\n",
|
|
"and a token level (text tagging and question answering).\n",
|
|
"As a concrete empirical case,\n",
|
|
"we will fine-tune BERT for natural language inference.\n",
|
|
"\n",
|
|
"As we have introduced in :numref:`sec_bert`,\n",
|
|
"BERT requires minimal architecture changes\n",
|
|
"for a wide range of natural language processing applications.\n",
|
|
"However, this benefit comes at the cost of fine-tuning\n",
|
|
"a huge number of BERT parameters for the downstream applications.\n",
|
|
"When space or time is limited,\n",
|
|
"those crafted models based on MLPs, CNNs, RNNs, and attention\n",
|
|
"are more feasible.\n",
|
|
"In the following, we start by the sentiment analysis application\n",
|
|
"and illustrate the model design based on RNNs and CNNs, respectively.\n",
|
|
"\n",
|
|
":begin_tab:toc\n",
|
|
" - [sentiment-analysis-and-dataset](sentiment-analysis-and-dataset.ipynb)\n",
|
|
" - [sentiment-analysis-rnn](sentiment-analysis-rnn.ipynb)\n",
|
|
" - [sentiment-analysis-cnn](sentiment-analysis-cnn.ipynb)\n",
|
|
" - [natural-language-inference-and-dataset](natural-language-inference-and-dataset.ipynb)\n",
|
|
" - [natural-language-inference-attention](natural-language-inference-attention.ipynb)\n",
|
|
" - [finetuning-bert](finetuning-bert.ipynb)\n",
|
|
" - [natural-language-inference-bert](natural-language-inference-bert.ipynb)\n",
|
|
":end_tab:\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"language_info": {
|
|
"name": "python"
|
|
},
|
|
"required_libs": []
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
} |