xixu-me/d2l-pytorch-notebooks
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
91fc65dcf00a1f997917edbb250edda5ff7bf4d7
d2l-pytorch-notebooks/chapter_appendix-mathematics-for-deep-learning/single-variable-calculus.ipynb
T
xixu-me 4c20ec342f Initial commit
2024-08-20 16:25:10 +08:00

4740 lines
190 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"cells": [
{
"cell_type": "markdown",
"id": "534548c2",
"metadata": {
"origin_pos": 0
},
"source": [
"# Single Variable Calculus\n",
":label:`sec_single_variable_calculus`\n",
"\n",
"In :numref:`sec_calculus`, we saw the basic elements of differential calculus. This section takes a deeper dive into the fundamentals of calculus and how we can understand and apply it in the context of machine learning.\n",
"\n",
"## Differential Calculus\n",
"Differential calculus is fundamentally the study of how functions behave under small changes. To see why this is so core to deep learning, let's consider an example.\n",
"\n",
"Suppose that we have a deep neural network where the weights are, for convenience, concatenated into a single vector $\\mathbf{w} = (w_1, \\ldots, w_n)$. Given a training dataset, we consider the loss of our neural network on this dataset, which we will write as $\\mathcal{L}(\\mathbf{w})$.\n",
"\n",
"This function is extraordinarily complex, encoding the performance of all possible models of the given architecture on this dataset, so it is nearly impossible to tell what set of weights $\\mathbf{w}$ will minimize the loss. Thus, in practice, we often start by initializing our weights *randomly*, and then iteratively take small steps in the direction which makes the loss decrease as rapidly as possible.\n",
"\n",
"The question then becomes something that on the surface is no easier: how do we find the direction which makes the weights decrease as quickly as possible? To dig into this, let's first examine the case with only a single weight: $L(\\mathbf{w}) = L(x)$ for a single real value $x$.\n",
"\n",
"Let's take $x$ and try to understand what happens when we change it by a small amount to $x + \\epsilon$. If you wish to be concrete, think a number like $\\epsilon = 0.0000001$. To help us visualize what happens, let's graph an example function, $f(x) = \\sin(x^x)$, over the $[0, 3]$.\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "b163c0f4",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T19:43:00.050382Z",
"iopub.status.busy": "2023-08-18T19:43:00.049957Z",
"iopub.status.idle": "2023-08-18T19:43:03.347341Z",
"shell.execute_reply": "2023-08-18T19:43:03.346361Z"
},
"origin_pos": 2,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"254.660938pt\" height=\"183.35625pt\" viewBox=\"0 0 254.660938 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T19:43:03.308381</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 183.35625 \n",
"L 254.660938 183.35625 \n",
"L 254.660938 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 52.160938 145.8 \n",
"L 247.460938 145.8 \n",
"L 247.460938 7.2 \n",
"L 52.160938 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 60.444413 145.8 \n",
"L 60.444413 7.2 \n",
"\" clip-path=\"url(#p9e1c05594e)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"ma87e072534\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#ma87e072534\" x=\"60.444413\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(57.263163 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 119.824163 145.8 \n",
"L 119.824163 7.2 \n",
"\" clip-path=\"url(#p9e1c05594e)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#ma87e072534\" x=\"119.824163\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(116.642913 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 179.203914 145.8 \n",
"L 179.203914 7.2 \n",
"\" clip-path=\"url(#p9e1c05594e)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#ma87e072534\" x=\"179.203914\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(176.022664 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 238.583665 145.8 \n",
"L 238.583665 7.2 \n",
"\" clip-path=\"url(#p9e1c05594e)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use xlink:href=\"#ma87e072534\" x=\"238.583665\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 3 -->\n",
" <g transform=\"translate(235.402415 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
"Q 3050 2419 3304 2112 \n",
"Q 3559 1806 3559 1356 \n",
"Q 3559 666 3084 287 \n",
"Q 2609 -91 1734 -91 \n",
"Q 1441 -91 1130 -33 \n",
"Q 819 25 488 141 \n",
"L 488 750 \n",
"Q 750 597 1062 519 \n",
"Q 1375 441 1716 441 \n",
"Q 2309 441 2620 675 \n",
"Q 2931 909 2931 1356 \n",
"Q 2931 1769 2642 2001 \n",
"Q 2353 2234 1838 2234 \n",
"L 1294 2234 \n",
"L 1294 2753 \n",
"L 1863 2753 \n",
"Q 2328 2753 2575 2939 \n",
"Q 2822 3125 2822 3475 \n",
"Q 2822 3834 2567 4026 \n",
"Q 2313 4219 1838 4219 \n",
"Q 1578 4219 1281 4162 \n",
"Q 984 4106 628 3988 \n",
"L 628 4550 \n",
"Q 988 4650 1302 4700 \n",
"Q 1616 4750 1894 4750 \n",
"Q 2613 4750 3031 4423 \n",
"Q 3450 4097 3450 3541 \n",
"Q 3450 3153 3228 2886 \n",
"Q 3006 2619 2597 2516 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-33\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- x -->\n",
" <g transform=\"translate(146.851563 174.076563) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 52.160938 139.543714 \n",
"L 247.460938 139.543714 \n",
"\" clip-path=\"url(#p9e1c05594e)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <defs>\n",
" <path id=\"m03514f5cb7\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m03514f5cb7\" x=\"52.160938\" y=\"139.543714\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −1.0 -->\n",
" <g transform=\"translate(20.878125 143.342933) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
"L 1344 794 \n",
"L 1344 0 \n",
"L 684 0 \n",
"L 684 794 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 52.160938 108.032046 \n",
"L 247.460938 108.032046 \n",
"\" clip-path=\"url(#p9e1c05594e)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <g>\n",
" <use xlink:href=\"#m03514f5cb7\" x=\"52.160938\" y=\"108.032046\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −0.5 -->\n",
" <g transform=\"translate(20.878125 111.831264) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
"L 3169 4666 \n",
"L 3169 4134 \n",
"L 1269 4134 \n",
"L 1269 2991 \n",
"Q 1406 3038 1543 3061 \n",
"Q 1681 3084 1819 3084 \n",
"Q 2600 3084 3056 2656 \n",
"Q 3513 2228 3513 1497 \n",
"Q 3513 744 3044 326 \n",
"Q 2575 -91 1722 -91 \n",
"Q 1428 -91 1123 -41 \n",
"Q 819 9 494 109 \n",
"L 494 744 \n",
"Q 775 591 1075 516 \n",
"Q 1375 441 1709 441 \n",
"Q 2250 441 2565 725 \n",
"Q 2881 1009 2881 1497 \n",
"Q 2881 1984 2565 2268 \n",
"Q 2250 2553 1709 2553 \n",
"Q 1456 2553 1204 2497 \n",
"Q 953 2441 691 2322 \n",
"L 691 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 52.160938 76.520377 \n",
"L 247.460938 76.520377 \n",
"\" clip-path=\"url(#p9e1c05594e)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m03514f5cb7\" x=\"52.160938\" y=\"76.520377\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0.0 -->\n",
" <g transform=\"translate(29.257812 80.319596) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 52.160938 45.008708 \n",
"L 247.460938 45.008708 \n",
"\" clip-path=\"url(#p9e1c05594e)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m03514f5cb7\" x=\"52.160938\" y=\"45.008708\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 0.5 -->\n",
" <g transform=\"translate(29.257812 48.807927) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 52.160938 13.49704 \n",
"L 247.460938 13.49704 \n",
"\" clip-path=\"url(#p9e1c05594e)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_18\">\n",
" <g>\n",
" <use xlink:href=\"#m03514f5cb7\" x=\"52.160938\" y=\"13.49704\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 1.0 -->\n",
" <g transform=\"translate(29.257812 17.296259) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- f(x) -->\n",
" <g transform=\"translate(14.798438 85.121094) rotate(-90) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
"L 2375 4384 \n",
"L 1825 4384 \n",
"Q 1516 4384 1395 4259 \n",
"Q 1275 4134 1275 3809 \n",
"L 1275 3500 \n",
"L 2222 3500 \n",
"L 2222 3053 \n",
"L 1275 3053 \n",
"L 1275 0 \n",
"L 697 0 \n",
"L 697 3053 \n",
"L 147 3053 \n",
"L 147 3500 \n",
"L 697 3500 \n",
"L 697 3744 \n",
"Q 697 4328 969 4595 \n",
"Q 1241 4863 1831 4863 \n",
"L 2375 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
"Q 1566 4138 1362 3434 \n",
"Q 1159 2731 1159 2009 \n",
"Q 1159 1288 1364 580 \n",
"Q 1569 -128 1984 -844 \n",
"L 1484 -844 \n",
"Q 1016 -109 783 600 \n",
"Q 550 1309 550 2009 \n",
"Q 550 2706 781 3412 \n",
"Q 1013 4119 1484 4856 \n",
"L 1984 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
"L 1013 4856 \n",
"Q 1481 4119 1714 3412 \n",
"Q 1947 2706 1947 2009 \n",
"Q 1947 1309 1714 600 \n",
"Q 1481 -109 1013 -844 \n",
"L 513 -844 \n",
"Q 928 -128 1133 580 \n",
"Q 1338 1288 1338 2009 \n",
"Q 1338 2731 1133 3434 \n",
"Q 928 4138 513 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-66\"/>\n",
" <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
" <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
" <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_19\">\n",
" <path d=\"M 61.03821 25.073831 \n",
"L 62.225805 27.146727 \n",
"L 63.4134 28.721967 \n",
"L 64.600995 30.007078 \n",
"L 66.382388 31.559971 \n",
"L 68.163781 32.782162 \n",
"L 69.945173 33.752353 \n",
"L 72.320363 34.737765 \n",
"L 74.695553 35.440235 \n",
"L 77.070743 35.911377 \n",
"L 79.445933 36.187234 \n",
"L 82.414921 36.296608 \n",
"L 85.383907 36.1782 \n",
"L 88.352895 35.858388 \n",
"L 91.915682 35.235634 \n",
"L 95.478464 34.37514 \n",
"L 99.041249 33.295549 \n",
"L 103.197831 31.779598 \n",
"L 107.354417 30.009633 \n",
"L 111.510999 28.012078 \n",
"L 116.261378 25.495726 \n",
"L 123.386945 21.427623 \n",
"L 129.918718 17.742595 \n",
"L 133.481507 15.965677 \n",
"L 136.450499 14.743216 \n",
"L 138.82568 14.013891 \n",
"L 140.60707 13.657089 \n",
"L 142.388468 13.500939 \n",
"L 144.169859 13.58389 \n",
"L 145.951257 13.948817 \n",
"L 147.138851 14.371996 \n",
"L 148.326445 14.95667 \n",
"L 149.514039 15.718884 \n",
"L 150.70164 16.67551 \n",
"L 151.889227 17.844122 \n",
"L 153.07682 19.242976 \n",
"L 154.264414 20.89075 \n",
"L 155.452015 22.806406 \n",
"L 157.233406 26.223484 \n",
"L 159.014797 30.347825 \n",
"L 160.796195 35.236213 \n",
"L 162.577586 40.93564 \n",
"L 164.358976 47.477496 \n",
"L 166.140367 54.870066 \n",
"L 167.921758 63.089407 \n",
"L 170.296953 75.210846 \n",
"L 173.265938 91.744569 \n",
"L 177.422516 115.218909 \n",
"L 179.203914 124.216596 \n",
"L 180.391508 129.436294 \n",
"L 181.579102 133.795907 \n",
"L 182.766696 137.076587 \n",
"L 183.360493 138.241792 \n",
"L 183.95429 139.05249 \n",
"L 184.548087 139.480876 \n",
"L 185.141883 139.5 \n",
"L 185.73568 139.084255 \n",
"L 186.329477 138.209881 \n",
"L 186.923274 136.855541 \n",
"L 187.517071 135.002899 \n",
"L 188.110868 132.637393 \n",
"L 189.298476 126.332105 \n",
"L 190.48607 117.92553 \n",
"L 191.673664 107.512186 \n",
"L 192.861258 95.321273 \n",
"L 194.642649 74.596504 \n",
"L 197.611633 39.175607 \n",
"L 198.799227 27.398186 \n",
"L 199.986821 18.632803 \n",
"L 200.580618 15.729233 \n",
"L 201.174415 13.984414 \n",
"L 201.768212 13.511022 \n",
"L 202.362009 14.401781 \n",
"L 202.955806 16.723769 \n",
"L 203.549617 20.512513 \n",
"L 204.143414 25.765734 \n",
"L 204.737211 32.438402 \n",
"L 205.924805 49.6168 \n",
"L 207.706196 81.966908 \n",
"L 209.487586 114.597846 \n",
"L 210.67518 130.998323 \n",
"L 211.268977 136.288751 \n",
"L 211.862774 139.13206 \n",
"L 212.456571 139.237719 \n",
"L 213.050368 136.42063 \n",
"L 213.644165 130.62799 \n",
"L 214.237962 121.963273 \n",
"L 214.831759 110.704348 \n",
"L 216.019353 82.435341 \n",
"L 217.800758 37.636061 \n",
"L 218.394555 26.03684 \n",
"L 218.988352 17.805043 \n",
"L 219.582149 13.7876 \n",
"L 220.175945 14.58201 \n",
"L 220.769742 20.442782 \n",
"L 221.363539 31.202576 \n",
"L 221.957336 46.220954 \n",
"L 224.332524 120.402601 \n",
"L 224.926321 132.856501 \n",
"L 225.520118 139.078079 \n",
"L 226.113915 137.895478 \n",
"L 226.707712 128.963897 \n",
"L 227.301509 112.948066 \n",
"L 228.489103 67.614778 \n",
"L 229.0829 44.465574 \n",
"L 229.676697 25.843516 \n",
"L 230.270493 15.114935 \n",
"L 230.86429 14.65134 \n",
"L 231.458087 25.222137 \n",
"L 232.051898 45.583546 \n",
"L 233.833289 124.303705 \n",
"L 234.427086 137.86182 \n",
"L 235.020883 137.457389 \n",
"L 235.61468 122.258638 \n",
"L 236.208477 95.149093 \n",
"L 237.396071 33.139163 \n",
"L 237.989868 15.677547 \n",
"L 238.583665 16.246373 \n",
"L 238.583665 16.246373 \n",
"\" clip-path=\"url(#p9e1c05594e)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 52.160938 145.8 \n",
"L 52.160938 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 247.460938 145.8 \n",
"L 247.460938 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 52.160938 145.8 \n",
"L 247.460938 145.8 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 52.160938 7.2 \n",
"L 247.460938 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p9e1c05594e\">\n",
" <rect x=\"52.160938\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 350x250 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%matplotlib inline\n",
"import torch\n",
"from IPython import display\n",
"from d2l import torch as d2l\n",
"\n",
"torch.pi = torch.acos(torch.zeros(1)).item() * 2 # Define pi in torch\n",
"\n",
"# Plot a function in a normal range\n",
"x_big = torch.arange(0.01, 3.01, 0.01)\n",
"ys = torch.sin(x_big**x_big)\n",
"d2l.plot(x_big, ys, 'x', 'f(x)')"
]
},
{
"cell_type": "markdown",
"id": "6541a198",
"metadata": {
"origin_pos": 4
},
"source": [
"At this large scale, the function's behavior is not simple. However, if we reduce our range to something smaller like $[1.75,2.25]$, we see that the graph becomes much simpler.\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "e6ee7e01",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T19:43:03.352193Z",
"iopub.status.busy": "2023-08-18T19:43:03.351414Z",
"iopub.status.idle": "2023-08-18T19:43:03.577808Z",
"shell.execute_reply": "2023-08-18T19:43:03.576947Z"
},
"origin_pos": 6,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"254.660938pt\" height=\"184.23422pt\" viewBox=\"0 0 254.660938 184.23422\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T19:43:03.539643</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 184.23422 \n",
"L 254.660938 184.23422 \n",
"L 254.660938 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 52.160938 146.67797 \n",
"L 247.460938 146.67797 \n",
"L 247.460938 8.07797 \n",
"L 52.160938 8.07797 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 78.828333 146.67797 \n",
"L 78.828333 8.07797 \n",
"\" clip-path=\"url(#p7e67761c9a)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"m217cb53d58\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m217cb53d58\" x=\"78.828333\" y=\"146.67797\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 1.8 -->\n",
" <g transform=\"translate(70.876771 161.276407) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
"L 1344 794 \n",
"L 1344 0 \n",
"L 684 0 \n",
"L 684 794 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
"Q 1584 2216 1326 1975 \n",
"Q 1069 1734 1069 1313 \n",
"Q 1069 891 1326 650 \n",
"Q 1584 409 2034 409 \n",
"Q 2484 409 2743 651 \n",
"Q 3003 894 3003 1313 \n",
"Q 3003 1734 2745 1975 \n",
"Q 2488 2216 2034 2216 \n",
"z\n",
"M 1403 2484 \n",
"Q 997 2584 770 2862 \n",
"Q 544 3141 544 3541 \n",
"Q 544 4100 942 4425 \n",
"Q 1341 4750 2034 4750 \n",
"Q 2731 4750 3128 4425 \n",
"Q 3525 4100 3525 3541 \n",
"Q 3525 3141 3298 2862 \n",
"Q 3072 2584 2669 2484 \n",
"Q 3125 2378 3379 2068 \n",
"Q 3634 1759 3634 1313 \n",
"Q 3634 634 3220 271 \n",
"Q 2806 -91 2034 -91 \n",
"Q 1263 -91 848 271 \n",
"Q 434 634 434 1313 \n",
"Q 434 1759 690 2068 \n",
"Q 947 2378 1403 2484 \n",
"z\n",
"M 1172 3481 \n",
"Q 1172 3119 1398 2916 \n",
"Q 1625 2713 2034 2713 \n",
"Q 2441 2713 2670 2916 \n",
"Q 2900 3119 2900 3481 \n",
"Q 2900 3844 2670 4047 \n",
"Q 2441 4250 2034 4250 \n",
"Q 1625 4250 1398 4047 \n",
"Q 1172 3844 1172 3481 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-38\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 114.40858 146.67797 \n",
"L 114.40858 8.07797 \n",
"\" clip-path=\"url(#p7e67761c9a)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#m217cb53d58\" x=\"114.40858\" y=\"146.67797\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 1.9 -->\n",
" <g transform=\"translate(106.457017 161.276407) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-39\" d=\"M 703 97 \n",
"L 703 672 \n",
"Q 941 559 1184 500 \n",
"Q 1428 441 1663 441 \n",
"Q 2288 441 2617 861 \n",
"Q 2947 1281 2994 2138 \n",
"Q 2813 1869 2534 1725 \n",
"Q 2256 1581 1919 1581 \n",
"Q 1219 1581 811 2004 \n",
"Q 403 2428 403 3163 \n",
"Q 403 3881 828 4315 \n",
"Q 1253 4750 1959 4750 \n",
"Q 2769 4750 3195 4129 \n",
"Q 3622 3509 3622 2328 \n",
"Q 3622 1225 3098 567 \n",
"Q 2575 -91 1691 -91 \n",
"Q 1453 -91 1209 -44 \n",
"Q 966 3 703 97 \n",
"z\n",
"M 1959 2075 \n",
"Q 2384 2075 2632 2365 \n",
"Q 2881 2656 2881 3163 \n",
"Q 2881 3666 2632 3958 \n",
"Q 2384 4250 1959 4250 \n",
"Q 1534 4250 1286 3958 \n",
"Q 1038 3666 1038 3163 \n",
"Q 1038 2656 1286 2365 \n",
"Q 1534 2075 1959 2075 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-39\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 149.988826 146.67797 \n",
"L 149.988826 8.07797 \n",
"\" clip-path=\"url(#p7e67761c9a)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#m217cb53d58\" x=\"149.988826\" y=\"146.67797\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 2.0 -->\n",
" <g transform=\"translate(142.037263 161.276407) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 185.569072 146.67797 \n",
"L 185.569072 8.07797 \n",
"\" clip-path=\"url(#p7e67761c9a)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use xlink:href=\"#m217cb53d58\" x=\"185.569072\" y=\"146.67797\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2.1 -->\n",
" <g transform=\"translate(177.61751 161.276407) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 221.149318 146.67797 \n",
"L 221.149318 8.07797 \n",
"\" clip-path=\"url(#p7e67761c9a)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use xlink:href=\"#m217cb53d58\" x=\"221.149318\" y=\"146.67797\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 2.2 -->\n",
" <g transform=\"translate(213.197756 161.276407) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- x -->\n",
" <g transform=\"translate(146.851563 174.954532) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 52.160938 140.378628 \n",
"L 247.460938 140.378628 \n",
"\" clip-path=\"url(#p7e67761c9a)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <defs>\n",
" <path id=\"m7d5d4bcd3e\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m7d5d4bcd3e\" x=\"52.160938\" y=\"140.378628\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −1.0 -->\n",
" <g transform=\"translate(20.878125 144.177847) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 52.160938 97.252158 \n",
"L 247.460938 97.252158 \n",
"\" clip-path=\"url(#p7e67761c9a)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m7d5d4bcd3e\" x=\"52.160938\" y=\"97.252158\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- −0.5 -->\n",
" <g transform=\"translate(20.878125 101.051377) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
"L 3169 4666 \n",
"L 3169 4134 \n",
"L 1269 4134 \n",
"L 1269 2991 \n",
"Q 1406 3038 1543 3061 \n",
"Q 1681 3084 1819 3084 \n",
"Q 2600 3084 3056 2656 \n",
"Q 3513 2228 3513 1497 \n",
"Q 3513 744 3044 326 \n",
"Q 2575 -91 1722 -91 \n",
"Q 1428 -91 1123 -41 \n",
"Q 819 9 494 109 \n",
"L 494 744 \n",
"Q 775 591 1075 516 \n",
"Q 1375 441 1709 441 \n",
"Q 2250 441 2565 725 \n",
"Q 2881 1009 2881 1497 \n",
"Q 2881 1984 2565 2268 \n",
"Q 2250 2553 1709 2553 \n",
"Q 1456 2553 1204 2497 \n",
"Q 953 2441 691 2322 \n",
"L 691 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 52.160938 54.125688 \n",
"L 247.460938 54.125688 \n",
"\" clip-path=\"url(#p7e67761c9a)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m7d5d4bcd3e\" x=\"52.160938\" y=\"54.125688\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 0.0 -->\n",
" <g transform=\"translate(29.257812 57.924907) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 52.160938 10.999219 \n",
"L 247.460938 10.999219 \n",
"\" clip-path=\"url(#p7e67761c9a)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_18\">\n",
" <g>\n",
" <use xlink:href=\"#m7d5d4bcd3e\" x=\"52.160938\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 0.5 -->\n",
" <g transform=\"translate(29.257812 14.798438) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- f(x) -->\n",
" <g transform=\"translate(14.798438 85.999064) rotate(-90) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
"L 2375 4384 \n",
"L 1825 4384 \n",
"Q 1516 4384 1395 4259 \n",
"Q 1275 4134 1275 3809 \n",
"L 1275 3500 \n",
"L 2222 3500 \n",
"L 2222 3053 \n",
"L 1275 3053 \n",
"L 1275 0 \n",
"L 697 0 \n",
"L 697 3053 \n",
"L 147 3053 \n",
"L 147 3500 \n",
"L 697 3500 \n",
"L 697 3744 \n",
"Q 697 4328 969 4595 \n",
"Q 1241 4863 1831 4863 \n",
"L 2375 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
"Q 1566 4138 1362 3434 \n",
"Q 1159 2731 1159 2009 \n",
"Q 1159 1288 1364 580 \n",
"Q 1569 -128 1984 -844 \n",
"L 1484 -844 \n",
"Q 1016 -109 783 600 \n",
"Q 550 1309 550 2009 \n",
"Q 550 2706 781 3412 \n",
"Q 1013 4119 1484 4856 \n",
"L 1984 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
"L 1013 4856 \n",
"Q 1481 4119 1714 3412 \n",
"Q 1947 2706 1947 2009 \n",
"Q 1947 1309 1714 600 \n",
"Q 1481 -109 1013 -844 \n",
"L 513 -844 \n",
"Q 928 -128 1133 580 \n",
"Q 1338 1288 1338 2009 \n",
"Q 1338 2731 1133 3434 \n",
"Q 928 4138 513 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-66\"/>\n",
" <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
" <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
" <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_19\">\n",
" <path d=\"M 61.03821 14.37797 \n",
"L 66.375242 19.291975 \n",
"L 71.712316 24.495386 \n",
"L 77.405125 30.358233 \n",
"L 83.097976 36.532566 \n",
"L 89.146603 43.415044 \n",
"L 95.55105 51.033508 \n",
"L 102.667093 59.844276 \n",
"L 111.206369 70.786954 \n",
"L 138.603124 106.216647 \n",
"L 144.295975 113.029395 \n",
"L 148.921411 118.246723 \n",
"L 153.19107 122.743951 \n",
"L 157.104868 126.545741 \n",
"L 160.66289 129.694506 \n",
"L 163.865134 132.246469 \n",
"L 167.067379 134.502651 \n",
"L 169.913762 136.237095 \n",
"L 172.76023 137.695799 \n",
"L 175.250751 138.730245 \n",
"L 177.741442 139.525112 \n",
"L 180.232049 140.067238 \n",
"L 182.366878 140.321048 \n",
"L 184.501708 140.371883 \n",
"L 186.636453 140.212129 \n",
"L 188.771283 139.834445 \n",
"L 190.906112 139.231849 \n",
"L 193.040942 138.397797 \n",
"L 195.175687 137.326243 \n",
"L 197.310517 136.011587 \n",
"L 199.445431 134.448896 \n",
"L 201.580176 132.634152 \n",
"L 204.070867 130.193524 \n",
"L 206.561389 127.401206 \n",
"L 209.051995 124.254682 \n",
"L 211.542602 120.753748 \n",
"L 214.033293 116.900131 \n",
"L 216.879676 112.069885 \n",
"L 219.726144 106.794745 \n",
"L 222.928303 100.347466 \n",
"L 226.130548 93.384815 \n",
"L 229.688569 85.090082 \n",
"L 233.602452 75.362004 \n",
"L 237.872027 64.151091 \n",
"L 238.583665 62.232723 \n",
"L 238.583665 62.232723 \n",
"\" clip-path=\"url(#p7e67761c9a)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 52.160938 146.67797 \n",
"L 52.160938 8.07797 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 247.460938 146.67797 \n",
"L 247.460938 8.07797 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 52.160938 146.67797 \n",
"L 247.460938 146.67797 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 52.160938 8.07797 \n",
"L 247.460938 8.07797 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p7e67761c9a\">\n",
" <rect x=\"52.160938\" y=\"8.07797\" width=\"195.3\" height=\"138.6\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 350x250 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot a the same function in a tiny range\n",
"x_med = torch.arange(1.75, 2.25, 0.001)\n",
"ys = torch.sin(x_med**x_med)\n",
"d2l.plot(x_med, ys, 'x', 'f(x)')"
]
},
{
"cell_type": "markdown",
"id": "48159f0d",
"metadata": {
"origin_pos": 8
},
"source": [
"Taking this to an extreme, if we zoom into a tiny segment, the behavior becomes far simpler: it is just a straight line.\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "2d612e9e",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T19:43:03.583975Z",
"iopub.status.busy": "2023-08-18T19:43:03.583628Z",
"iopub.status.idle": "2023-08-18T19:43:03.845892Z",
"shell.execute_reply": "2023-08-18T19:43:03.844812Z"
},
"origin_pos": 10,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"268.25193pt\" height=\"183.35625pt\" viewBox=\"0 0 268.25193 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T19:43:03.801080</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 183.35625 \n",
"L 268.25193 183.35625 \n",
"L 268.25193 -0 \n",
"L 0 -0 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 58.523438 145.8 \n",
"L 253.823438 145.8 \n",
"L 253.823438 7.2 \n",
"L 58.523438 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 67.40071 145.8 \n",
"L 67.40071 7.2 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"m28dbb00b42\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m28dbb00b42\" x=\"67.40071\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 2.000 -->\n",
" <g transform=\"translate(53.086648 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
"L 1344 794 \n",
"L 1344 0 \n",
"L 684 0 \n",
"L 684 794 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"159.033203\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"222.65625\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 103.268142 145.8 \n",
"L 103.268142 7.2 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#m28dbb00b42\" x=\"103.268142\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 2.002 -->\n",
" <g transform=\"translate(88.954079 160.398438) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"159.033203\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"222.65625\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 139.135573 145.8 \n",
"L 139.135573 7.2 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#m28dbb00b42\" x=\"139.135573\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 2.004 -->\n",
" <g transform=\"translate(124.82151 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
"L 825 1625 \n",
"L 2419 1625 \n",
"L 2419 4116 \n",
"z\n",
"M 2253 4666 \n",
"L 3047 4666 \n",
"L 3047 1625 \n",
"L 3713 1625 \n",
"L 3713 1100 \n",
"L 3047 1100 \n",
"L 3047 0 \n",
"L 2419 0 \n",
"L 2419 1100 \n",
"L 313 1100 \n",
"L 313 1709 \n",
"L 2253 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"159.033203\"/>\n",
" <use xlink:href=\"#DejaVuSans-34\" x=\"222.65625\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 175.003004 145.8 \n",
"L 175.003004 7.2 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use xlink:href=\"#m28dbb00b42\" x=\"175.003004\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 2.006 -->\n",
" <g transform=\"translate(160.688942 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
"Q 1688 2584 1439 2293 \n",
"Q 1191 2003 1191 1497 \n",
"Q 1191 994 1439 701 \n",
"Q 1688 409 2113 409 \n",
"Q 2538 409 2786 701 \n",
"Q 3034 994 3034 1497 \n",
"Q 3034 2003 2786 2293 \n",
"Q 2538 2584 2113 2584 \n",
"z\n",
"M 3366 4563 \n",
"L 3366 3988 \n",
"Q 3128 4100 2886 4159 \n",
"Q 2644 4219 2406 4219 \n",
"Q 1781 4219 1451 3797 \n",
"Q 1122 3375 1075 2522 \n",
"Q 1259 2794 1537 2939 \n",
"Q 1816 3084 2150 3084 \n",
"Q 2853 3084 3261 2657 \n",
"Q 3669 2231 3669 1497 \n",
"Q 3669 778 3244 343 \n",
"Q 2819 -91 2113 -91 \n",
"Q 1303 -91 875 529 \n",
"Q 447 1150 447 2328 \n",
"Q 447 3434 972 4092 \n",
"Q 1497 4750 2381 4750 \n",
"Q 2619 4750 2861 4703 \n",
"Q 3103 4656 3366 4563 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"159.033203\"/>\n",
" <use xlink:href=\"#DejaVuSans-36\" x=\"222.65625\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 210.870436 145.8 \n",
"L 210.870436 7.2 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use xlink:href=\"#m28dbb00b42\" x=\"210.870436\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 2.008 -->\n",
" <g transform=\"translate(196.556373 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
"Q 1584 2216 1326 1975 \n",
"Q 1069 1734 1069 1313 \n",
"Q 1069 891 1326 650 \n",
"Q 1584 409 2034 409 \n",
"Q 2484 409 2743 651 \n",
"Q 3003 894 3003 1313 \n",
"Q 3003 1734 2745 1975 \n",
"Q 2488 2216 2034 2216 \n",
"z\n",
"M 1403 2484 \n",
"Q 997 2584 770 2862 \n",
"Q 544 3141 544 3541 \n",
"Q 544 4100 942 4425 \n",
"Q 1341 4750 2034 4750 \n",
"Q 2731 4750 3128 4425 \n",
"Q 3525 4100 3525 3541 \n",
"Q 3525 3141 3298 2862 \n",
"Q 3072 2584 2669 2484 \n",
"Q 3125 2378 3379 2068 \n",
"Q 3634 1759 3634 1313 \n",
"Q 3634 634 3220 271 \n",
"Q 2806 -91 2034 -91 \n",
"Q 1263 -91 848 271 \n",
"Q 434 634 434 1313 \n",
"Q 434 1759 690 2068 \n",
"Q 947 2378 1403 2484 \n",
"z\n",
"M 1172 3481 \n",
"Q 1172 3119 1398 2916 \n",
"Q 1625 2713 2034 2713 \n",
"Q 2441 2713 2670 2916 \n",
"Q 2900 3119 2900 3481 \n",
"Q 2900 3844 2670 4047 \n",
"Q 2441 4250 2034 4250 \n",
"Q 1625 4250 1398 4047 \n",
"Q 1172 3844 1172 3481 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"159.033203\"/>\n",
" <use xlink:href=\"#DejaVuSans-38\" x=\"222.65625\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_6\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 246.737867 145.8 \n",
"L 246.737867 7.2 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <g>\n",
" <use xlink:href=\"#m28dbb00b42\" x=\"246.737867\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- 2.010 -->\n",
" <g transform=\"translate(232.423805 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"159.033203\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"222.65625\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- x -->\n",
" <g transform=\"translate(153.214063 174.076563) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 58.523438 141.590642 \n",
"L 253.823438 141.590642 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <defs>\n",
" <path id=\"m03f6b7d571\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m03f6b7d571\" x=\"58.523438\" y=\"141.590642\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- −0.80 -->\n",
" <g transform=\"translate(20.878125 145.38986) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-38\" x=\"179.199219\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"242.822266\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 58.523438 111.938315 \n",
"L 253.823438 111.938315 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m03f6b7d571\" x=\"58.523438\" y=\"111.938315\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- −0.79 -->\n",
" <g transform=\"translate(20.878125 115.737533) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-37\" d=\"M 525 4666 \n",
"L 3525 4666 \n",
"L 3525 4397 \n",
"L 1831 0 \n",
"L 1172 0 \n",
"L 2766 4134 \n",
"L 525 4134 \n",
"L 525 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-39\" d=\"M 703 97 \n",
"L 703 672 \n",
"Q 941 559 1184 500 \n",
"Q 1428 441 1663 441 \n",
"Q 2288 441 2617 861 \n",
"Q 2947 1281 2994 2138 \n",
"Q 2813 1869 2534 1725 \n",
"Q 2256 1581 1919 1581 \n",
"Q 1219 1581 811 2004 \n",
"Q 403 2428 403 3163 \n",
"Q 403 3881 828 4315 \n",
"Q 1253 4750 1959 4750 \n",
"Q 2769 4750 3195 4129 \n",
"Q 3622 3509 3622 2328 \n",
"Q 3622 1225 3098 567 \n",
"Q 2575 -91 1691 -91 \n",
"Q 1453 -91 1209 -44 \n",
"Q 966 3 703 97 \n",
"z\n",
"M 1959 2075 \n",
"Q 2384 2075 2632 2365 \n",
"Q 2881 2656 2881 3163 \n",
"Q 2881 3666 2632 3958 \n",
"Q 2384 4250 1959 4250 \n",
"Q 1534 4250 1286 3958 \n",
"Q 1038 3666 1038 3163 \n",
"Q 1038 2656 1286 2365 \n",
"Q 1534 2075 1959 2075 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-37\" x=\"179.199219\"/>\n",
" <use xlink:href=\"#DejaVuSans-39\" x=\"242.822266\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 58.523438 82.285988 \n",
"L 253.823438 82.285988 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_18\">\n",
" <g>\n",
" <use xlink:href=\"#m03f6b7d571\" x=\"58.523438\" y=\"82.285988\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- −0.78 -->\n",
" <g transform=\"translate(20.878125 86.085206) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-37\" x=\"179.199219\"/>\n",
" <use xlink:href=\"#DejaVuSans-38\" x=\"242.822266\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_19\">\n",
" <path d=\"M 58.523438 52.633661 \n",
"L 253.823438 52.633661 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_20\">\n",
" <g>\n",
" <use xlink:href=\"#m03f6b7d571\" x=\"58.523438\" y=\"52.633661\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- −0.77 -->\n",
" <g transform=\"translate(20.878125 56.432879) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-37\" x=\"179.199219\"/>\n",
" <use xlink:href=\"#DejaVuSans-37\" x=\"242.822266\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_21\">\n",
" <path d=\"M 58.523438 22.981334 \n",
"L 253.823438 22.981334 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_22\">\n",
" <g>\n",
" <use xlink:href=\"#m03f6b7d571\" x=\"58.523438\" y=\"22.981334\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_12\">\n",
" <!-- −0.76 -->\n",
" <g transform=\"translate(20.878125 26.780552) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-37\" x=\"179.199219\"/>\n",
" <use xlink:href=\"#DejaVuSans-36\" x=\"242.822266\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_13\">\n",
" <!-- f(x) -->\n",
" <g transform=\"translate(14.798437 85.121094) rotate(-90) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
"L 2375 4384 \n",
"L 1825 4384 \n",
"Q 1516 4384 1395 4259 \n",
"Q 1275 4134 1275 3809 \n",
"L 1275 3500 \n",
"L 2222 3500 \n",
"L 2222 3053 \n",
"L 1275 3053 \n",
"L 1275 0 \n",
"L 697 0 \n",
"L 697 3053 \n",
"L 147 3053 \n",
"L 147 3500 \n",
"L 697 3500 \n",
"L 697 3744 \n",
"Q 697 4328 969 4595 \n",
"Q 1241 4863 1831 4863 \n",
"L 2375 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
"Q 1566 4138 1362 3434 \n",
"Q 1159 2731 1159 2009 \n",
"Q 1159 1288 1364 580 \n",
"Q 1569 -128 1984 -844 \n",
"L 1484 -844 \n",
"Q 1016 -109 783 600 \n",
"Q 550 1309 550 2009 \n",
"Q 550 2706 781 3412 \n",
"Q 1013 4119 1484 4856 \n",
"L 1984 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
"L 1013 4856 \n",
"Q 1481 4119 1714 3412 \n",
"Q 1947 2706 1947 2009 \n",
"Q 1947 1309 1714 600 \n",
"Q 1481 -109 1013 -844 \n",
"L 513 -844 \n",
"Q 928 -128 1133 580 \n",
"Q 1338 1288 1338 2009 \n",
"Q 1338 2731 1133 3434 \n",
"Q 928 4138 513 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-66\"/>\n",
" <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
" <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
" <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_23\">\n",
" <path d=\"M 67.40071 13.5 \n",
"L 69.192242 14.810893 \n",
"L 70.988049 16.124437 \n",
"L 72.77958 17.433385 \n",
"L 74.575387 18.744985 \n",
"L 76.366918 20.05358 \n",
"L 78.162725 21.363059 \n",
"L 79.954256 22.66971 \n",
"L 81.745788 23.975124 \n",
"L 83.537319 25.279654 \n",
"L 85.333126 26.586835 \n",
"L 87.124657 27.890304 \n",
"L 88.920464 29.196248 \n",
"L 90.711996 30.49742 \n",
"L 92.507803 31.80142 \n",
"L 94.299334 33.101532 \n",
"L 96.095141 34.404294 \n",
"L 97.886672 35.702285 \n",
"L 99.682479 37.002927 \n",
"L 101.474011 38.29968 \n",
"L 103.269818 39.599085 \n",
"L 105.061349 40.894778 \n",
"L 106.857156 42.192061 \n",
"L 108.648687 43.485633 \n",
"L 110.440219 44.779029 \n",
"L 112.23175 46.071364 \n",
"L 114.027557 47.365466 \n",
"L 115.819088 48.656564 \n",
"L 117.614895 49.949429 \n",
"L 119.406427 51.237522 \n",
"L 121.202234 52.52915 \n",
"L 122.993765 53.817066 \n",
"L 124.789572 55.106396 \n",
"L 126.581103 56.392192 \n",
"L 128.37691 57.680462 \n",
"L 130.168441 58.963959 \n",
"L 131.964249 60.250992 \n",
"L 133.75578 61.533429 \n",
"L 135.551587 62.818164 \n",
"L 137.343118 64.098481 \n",
"L 139.134649 65.378444 \n",
"L 140.926181 66.658407 \n",
"L 142.721988 67.93996 \n",
"L 144.513519 69.217626 \n",
"L 146.309326 70.496882 \n",
"L 148.100857 71.773487 \n",
"L 149.896664 73.051505 \n",
"L 151.688196 74.325813 \n",
"L 153.484003 75.602594 \n",
"L 155.275534 76.874781 \n",
"L 157.071341 78.150325 \n",
"L 158.862872 79.421274 \n",
"L 160.658679 80.694698 \n",
"L 162.450211 81.963349 \n",
"L 164.246018 83.235536 \n",
"L 166.037549 84.50295 \n",
"L 167.82908 85.770187 \n",
"L 169.620612 87.036365 \n",
"L 171.416419 88.304132 \n",
"L 173.20795 89.569072 \n",
"L 175.003757 90.835426 \n",
"L 176.795288 92.098245 \n",
"L 178.591095 93.363362 \n",
"L 180.382626 94.623883 \n",
"L 182.178434 95.887763 \n",
"L 183.969965 97.14687 \n",
"L 185.765772 98.408452 \n",
"L 187.557303 99.666499 \n",
"L 189.35311 100.92596 \n",
"L 191.144641 102.181533 \n",
"L 192.940448 103.439756 \n",
"L 194.73198 104.694092 \n",
"L 196.523511 105.947367 \n",
"L 198.315042 107.200465 \n",
"L 200.110849 108.454977 \n",
"L 201.902381 109.705778 \n",
"L 203.698188 110.959053 \n",
"L 205.489719 112.208616 \n",
"L 207.285526 113.459593 \n",
"L 209.077057 114.706859 \n",
"L 210.868589 115.953948 \n",
"L 212.66012 117.198916 \n",
"L 214.455927 118.447243 \n",
"L 216.247458 119.690974 \n",
"L 218.043265 120.937002 \n",
"L 219.834797 122.179319 \n",
"L 221.630604 123.424111 \n",
"L 223.422135 124.66413 \n",
"L 225.217942 125.907507 \n",
"L 227.009473 127.146113 \n",
"L 228.80528 128.386309 \n",
"L 230.596811 129.623677 \n",
"L 232.392619 130.862459 \n",
"L 234.18415 132.098414 \n",
"L 235.979957 133.335959 \n",
"L 237.771488 134.568731 \n",
"L 239.563019 135.801328 \n",
"L 241.358826 137.036222 \n",
"L 243.150358 138.267404 \n",
"L 244.946165 139.5 \n",
"\" clip-path=\"url(#p8940136e99)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 58.523438 145.8 \n",
"L 58.523438 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 253.823438 145.8 \n",
"L 253.823438 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 58.523438 145.8 \n",
"L 253.823438 145.8 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 58.523438 7.2 \n",
"L 253.823438 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p8940136e99\">\n",
" <rect x=\"58.523438\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 350x250 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot a the same function in a tiny range\n",
"x_small = torch.arange(2.0, 2.01, 0.0001)\n",
"ys = torch.sin(x_small**x_small)\n",
"d2l.plot(x_small, ys, 'x', 'f(x)')"
]
},
{
"cell_type": "markdown",
"id": "95c80784",
"metadata": {
"origin_pos": 12
},
"source": [
"This is the key observation of single variable calculus: the behavior of familiar functions can be modeled by a line in a small enough range. This means that for most functions, it is reasonable to expect that as we shift the $x$ value of the function by a little bit, the output $f(x)$ will also be shifted by a little bit. The only question we need to answer is, \"How large is the change in the output compared to the change in the input? Is it half as large? Twice as large?\"\n",
"\n",
"Thus, we can consider the ratio of the change in the output of a function for a small change in the input of the function. We can write this formally as\n",
"\n",
"$$\n",
"\\frac{L(x+\\epsilon) - L(x)}{(x+\\epsilon) - x} = \\frac{L(x+\\epsilon) - L(x)}{\\epsilon}.\n",
"$$\n",
"\n",
"This is already enough to start to play around with in code. For instance, suppose that we know that $L(x) = x^{2} + 1701(x-4)^3$, then we can see how large this value is at the point $x = 4$ as follows.\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "35fc4965",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T19:43:03.849977Z",
"iopub.status.busy": "2023-08-18T19:43:03.849400Z",
"iopub.status.idle": "2023-08-18T19:43:03.854621Z",
"shell.execute_reply": "2023-08-18T19:43:03.853702Z"
},
"origin_pos": 13,
"tab": [
"pytorch"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"epsilon = 0.10000 -> 25.11000\n",
"epsilon = 0.00100 -> 8.00270\n",
"epsilon = 0.00010 -> 8.00012\n",
"epsilon = 0.00001 -> 8.00001\n"
]
}
],
"source": [
"# Define our function\n",
"def L(x):\n",
" return x**2 + 1701*(x-4)**3\n",
"\n",
"# Print the difference divided by epsilon for several epsilon\n",
"for epsilon in [0.1, 0.001, 0.0001, 0.00001]:\n",
" print(f'epsilon = {epsilon:.5f} -> {(L(4+epsilon) - L(4)) / epsilon:.5f}')"
]
},
{
"cell_type": "markdown",
"id": "fecab426",
"metadata": {
"origin_pos": 14
},
"source": [
"Now, if we are observant, we will notice that the output of this number is suspiciously close to $8$. Indeed, if we decrease $\\epsilon$, we will see value becomes progressively closer to $8$. Thus we may conclude, correctly, that the value we seek (the degree a change in the input changes the output) should be $8$ at the point $x=4$. The way that a mathematician encodes this fact is\n",
"\n",
"$$\n",
"\\lim_{\\epsilon \\rightarrow 0}\\frac{L(4+\\epsilon) - L(4)}{\\epsilon} = 8.\n",
"$$\n",
"\n",
"As a bit of a historical digression: in the first few decades of neural network research, scientists used this algorithm (the *method of finite differences*) to evaluate how a loss function changed under small perturbation: just change the weights and see how the loss changed. This is computationally inefficient, requiring two evaluations of the loss function to see how a single change of one variable influenced the loss. If we tried to do this with even a paltry few thousand parameters, it would require several thousand evaluations of the network over the entire dataset! It was not solved until 1986 that the *backpropagation algorithm* introduced in :citet:`Rumelhart.Hinton.Williams.ea.1988` provided a way to calculate how *any* change of the weights together would change the loss in the same computation time as a single prediction of the network over the dataset.\n",
"\n",
"Back in our example, this value $8$ is different for different values of $x$, so it makes sense to define it as a function of $x$. More formally, this value dependent rate of change is referred to as the *derivative* which is written as\n",
"\n",
"$$\\frac{df}{dx}(x) = \\lim_{\\epsilon \\rightarrow 0}\\frac{f(x+\\epsilon) - f(x)}{\\epsilon}.$$\n",
":eqlabel:`eq_der_def`\n",
"\n",
"Different texts will use different notations for the derivative. For instance, all of the below notations indicate the same thing:\n",
"\n",
"$$\n",
"\\frac{df}{dx} = \\frac{d}{dx}f = f' = \\nabla_xf = D_xf = f_x.\n",
"$$\n",
"\n",
"Most authors will pick a single notation and stick with it, however even that is not guaranteed. It is best to be familiar with all of these. We will use the notation $\\frac{df}{dx}$ throughout this text, unless we want to take the derivative of a complex expression, in which case we will use $\\frac{d}{dx}f$ to write expressions like\n",
"$$\n",
"\\frac{d}{dx}\\left[x^4+\\cos\\left(\\frac{x^2+1}{2x-1}\\right)\\right].\n",
"$$\n",
"\n",
"Oftentimes, it is intuitively useful to unravel the definition of derivative :eqref:`eq_der_def` again to see how a function changes when we make a small change of $x$:\n",
"\n",
"$$\\begin{aligned} \\frac{df}{dx}(x) = \\lim_{\\epsilon \\rightarrow 0}\\frac{f(x+\\epsilon) - f(x)}{\\epsilon} & \\implies \\frac{df}{dx}(x) \\approx \\frac{f(x+\\epsilon) - f(x)}{\\epsilon} \\\\ & \\implies \\epsilon \\frac{df}{dx}(x) \\approx f(x+\\epsilon) - f(x) \\\\ & \\implies f(x+\\epsilon) \\approx f(x) + \\epsilon \\frac{df}{dx}(x). \\end{aligned}$$\n",
":eqlabel:`eq_small_change`\n",
"\n",
"The last equation is worth explicitly calling out. It tells us that if you take any function and change the input by a small amount, the output would change by that small amount scaled by the derivative.\n",
"\n",
"In this way, we can understand the derivative as the scaling factor that tells us how large of change we get in the output from a change in the input.\n",
"\n",
"## Rules of Calculus\n",
":label:`sec_derivative_table`\n",
"\n",
"We now turn to the task of understanding how to compute the derivative of an explicit function. A full formal treatment of calculus would derive everything from first principles. We will not indulge in this temptation here, but rather provide an understanding of the common rules encountered.\n",
"\n",
"### Common Derivatives\n",
"As was seen in :numref:`sec_calculus`, when computing derivatives one can oftentimes use a series of rules to reduce the computation to a few core functions. We repeat them here for ease of reference.\n",
"\n",
"* **Derivative of constants.** $\\frac{d}{dx}c = 0$.\n",
"* **Derivative of linear functions.** $\\frac{d}{dx}(ax) = a$.\n",
"* **Power rule.** $\\frac{d}{dx}x^n = nx^{n-1}$.\n",
"* **Derivative of exponentials.** $\\frac{d}{dx}e^x = e^x$.\n",
"* **Derivative of the logarithm.** $\\frac{d}{dx}\\log(x) = \\frac{1}{x}$.\n",
"\n",
"### Derivative Rules\n",
"If every derivative needed to be separately computed and stored in a table, differential calculus would be near impossible. It is a gift of mathematics that we can generalize the above derivatives and compute more complex derivatives like finding the derivative of $f(x) = \\log\\left(1+(x-1)^{10}\\right)$. As was mentioned in :numref:`sec_calculus`, the key to doing so is to codify what happens when we take functions and combine them in various ways, most importantly: sums, products, and compositions.\n",
"\n",
"* **Sum rule.** $\\frac{d}{dx}\\left(g(x) + h(x)\\right) = \\frac{dg}{dx}(x) + \\frac{dh}{dx}(x)$.\n",
"* **Product rule.** $\\frac{d}{dx}\\left(g(x)\\cdot h(x)\\right) = g(x)\\frac{dh}{dx}(x) + \\frac{dg}{dx}(x)h(x)$.\n",
"* **Chain rule.** $\\frac{d}{dx}g(h(x)) = \\frac{dg}{dh}(h(x))\\cdot \\frac{dh}{dx}(x)$.\n",
"\n",
"Let's see how we may use :eqref:`eq_small_change` to understand these rules. For the sum rule, consider following chain of reasoning:\n",
"\n",
"$$\n",
"\\begin{aligned}\n",
"f(x+\\epsilon) & = g(x+\\epsilon) + h(x+\\epsilon) \\\\\n",
"& \\approx g(x) + \\epsilon \\frac{dg}{dx}(x) + h(x) + \\epsilon \\frac{dh}{dx}(x) \\\\\n",
"& = g(x) + h(x) + \\epsilon\\left(\\frac{dg}{dx}(x) + \\frac{dh}{dx}(x)\\right) \\\\\n",
"& = f(x) + \\epsilon\\left(\\frac{dg}{dx}(x) + \\frac{dh}{dx}(x)\\right).\n",
"\\end{aligned}\n",
"$$\n",
"\n",
"By comparing this result with the fact that $f(x+\\epsilon) \\approx f(x) + \\epsilon \\frac{df}{dx}(x)$, we see that $\\frac{df}{dx}(x) = \\frac{dg}{dx}(x) + \\frac{dh}{dx}(x)$ as desired. The intuition here is: when we change the input $x$, $g$ and $h$ jointly contribute to the change of the output by $\\frac{dg}{dx}(x)$ and $\\frac{dh}{dx}(x)$.\n",
"\n",
"\n",
"The product is more subtle, and will require a new observation about how to work with these expressions. We will begin as before using :eqref:`eq_small_change`:\n",
"\n",
"$$\n",
"\\begin{aligned}\n",
"f(x+\\epsilon) & = g(x+\\epsilon)\\cdot h(x+\\epsilon) \\\\\n",
"& \\approx \\left(g(x) + \\epsilon \\frac{dg}{dx}(x)\\right)\\cdot\\left(h(x) + \\epsilon \\frac{dh}{dx}(x)\\right) \\\\\n",
"& = g(x)\\cdot h(x) + \\epsilon\\left(g(x)\\frac{dh}{dx}(x) + \\frac{dg}{dx}(x)h(x)\\right) + \\epsilon^2\\frac{dg}{dx}(x)\\frac{dh}{dx}(x) \\\\\n",
"& = f(x) + \\epsilon\\left(g(x)\\frac{dh}{dx}(x) + \\frac{dg}{dx}(x)h(x)\\right) + \\epsilon^2\\frac{dg}{dx}(x)\\frac{dh}{dx}(x). \\\\\n",
"\\end{aligned}\n",
"$$\n",
"\n",
"\n",
"This resembles the computation done above, and indeed we see our answer ($\\frac{df}{dx}(x) = g(x)\\frac{dh}{dx}(x) + \\frac{dg}{dx}(x)h(x)$) sitting next to $\\epsilon$, but there is the issue of that term of size $\\epsilon^{2}$. We will refer to this as a *higher-order term*, since the power of $\\epsilon^2$ is higher than the power of $\\epsilon^1$. We will see in a later section that we will sometimes want to keep track of these, however for now observe that if $\\epsilon = 0.0000001$, then $\\epsilon^{2}= 0.0000000000001$, which is vastly smaller. As we send $\\epsilon \\rightarrow 0$, we may safely ignore the higher order terms. As a general convention in this appendix, we will use \"$\\approx$\" to denote that the two terms are equal up to higher order terms. However, if we wish to be more formal we may examine the difference quotient\n",
"\n",
"$$\n",
"\\frac{f(x+\\epsilon) - f(x)}{\\epsilon} = g(x)\\frac{dh}{dx}(x) + \\frac{dg}{dx}(x)h(x) + \\epsilon \\frac{dg}{dx}(x)\\frac{dh}{dx}(x),\n",
"$$\n",
"\n",
"and see that as we send $\\epsilon \\rightarrow 0$, the right hand term goes to zero as well.\n",
"\n",
"Finally, with the chain rule, we can again progress as before using :eqref:`eq_small_change` and see that\n",
"\n",
"$$\n",
"\\begin{aligned}\n",
"f(x+\\epsilon) & = g(h(x+\\epsilon)) \\\\\n",
"& \\approx g\\left(h(x) + \\epsilon \\frac{dh}{dx}(x)\\right) \\\\\n",
"& \\approx g(h(x)) + \\epsilon \\frac{dh}{dx}(x) \\frac{dg}{dh}(h(x))\\\\\n",
"& = f(x) + \\epsilon \\frac{dg}{dh}(h(x))\\frac{dh}{dx}(x),\n",
"\\end{aligned}\n",
"$$\n",
"\n",
"where in the second line we view the function $g$ as having its input ($h(x)$) shifted by the tiny quantity $\\epsilon \\frac{dh}{dx}(x)$.\n",
"\n",
"These rule provide us with a flexible set of tools to compute essentially any expression desired. For instance,\n",
"\n",
"$$\n",
"\\begin{aligned}\n",
"\\frac{d}{dx}\\left[\\log\\left(1+(x-1)^{10}\\right)\\right] & = \\left(1+(x-1)^{10}\\right)^{-1}\\frac{d}{dx}\\left[1+(x-1)^{10}\\right]\\\\\n",
"& = \\left(1+(x-1)^{10}\\right)^{-1}\\left(\\frac{d}{dx}[1] + \\frac{d}{dx}[(x-1)^{10}]\\right) \\\\\n",
"& = \\left(1+(x-1)^{10}\\right)^{-1}\\left(0 + 10(x-1)^9\\frac{d}{dx}[x-1]\\right) \\\\\n",
"& = 10\\left(1+(x-1)^{10}\\right)^{-1}(x-1)^9 \\\\\n",
"& = \\frac{10(x-1)^9}{1+(x-1)^{10}}.\n",
"\\end{aligned}\n",
"$$\n",
"\n",
"Where each line has used the following rules:\n",
"\n",
"1. The chain rule and derivative of logarithm.\n",
"2. The sum rule.\n",
"3. The derivative of constants, chain rule, and power rule.\n",
"4. The sum rule, derivative of linear functions, derivative of constants.\n",
"\n",
"Two things should be clear after doing this example:\n",
"\n",
"1. Any function we can write down using sums, products, constants, powers, exponentials, and logarithms can have its derivate computed mechanically by following these rules.\n",
"2. Having a human follow these rules can be tedious and error prone!\n",
"\n",
"Thankfully, these two facts together hint towards a way forward: this is a perfect candidate for mechanization! Indeed backpropagation, which we will revisit later in this section, is exactly that.\n",
"\n",
"### Linear Approximation\n",
"When working with derivatives, it is often useful to geometrically interpret the approximation used above. In particular, note that the equation\n",
"\n",
"$$\n",
"f(x+\\epsilon) \\approx f(x) + \\epsilon \\frac{df}{dx}(x),\n",
"$$\n",
"\n",
"approximates the value of $f$ by a line which passes through the point $(x, f(x))$ and has slope $\\frac{df}{dx}(x)$. In this way we say that the derivative gives a linear approximation to the function $f$, as illustrated below:\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "53dd0dc7",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T19:43:03.858490Z",
"iopub.status.busy": "2023-08-18T19:43:03.857907Z",
"iopub.status.idle": "2023-08-18T19:43:04.340403Z",
"shell.execute_reply": "2023-08-18T19:43:04.339367Z"
},
"origin_pos": 16,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"254.660937pt\" height=\"187.155469pt\" viewBox=\"0 0 254.660937 187.155469\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T19:43:04.294665</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 187.155469 \n",
"L 254.660937 187.155469 \n",
"L 254.660937 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 52.160938 149.599219 \n",
"L 247.460938 149.599219 \n",
"L 247.460938 10.999219 \n",
"L 52.160938 10.999219 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 93.312829 149.599219 \n",
"L 93.312829 10.999219 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"m446e8cb83c\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m446e8cb83c\" x=\"93.312829\" y=\"149.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- −2 -->\n",
" <g transform=\"translate(85.941735 164.197656) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 149.855967 149.599219 \n",
"L 149.855967 10.999219 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#m446e8cb83c\" x=\"149.855967\" y=\"149.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(146.674717 164.197656) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 206.399106 149.599219 \n",
"L 206.399106 10.999219 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#m446e8cb83c\" x=\"206.399106\" y=\"149.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(203.217856 164.197656) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- x -->\n",
" <g transform=\"translate(146.851563 177.875781) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 52.160938 149.599219 \n",
"L 247.460938 149.599219 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <defs>\n",
" <path id=\"m144608ae0a\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m144608ae0a\" x=\"52.160938\" y=\"149.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- −1.5 -->\n",
" <g transform=\"translate(20.878125 153.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
"L 1344 794 \n",
"L 1344 0 \n",
"L 684 0 \n",
"L 684 794 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
"L 3169 4666 \n",
"L 3169 4134 \n",
"L 1269 4134 \n",
"L 1269 2991 \n",
"Q 1406 3038 1543 3061 \n",
"Q 1681 3084 1819 3084 \n",
"Q 2600 3084 3056 2656 \n",
"Q 3513 2228 3513 1497 \n",
"Q 3513 744 3044 326 \n",
"Q 2575 -91 1722 -91 \n",
"Q 1428 -91 1123 -41 \n",
"Q 819 9 494 109 \n",
"L 494 744 \n",
"Q 775 591 1075 516 \n",
"Q 1375 441 1709 441 \n",
"Q 2250 441 2565 725 \n",
"Q 2881 1009 2881 1497 \n",
"Q 2881 1984 2565 2268 \n",
"Q 2250 2553 1709 2553 \n",
"Q 1456 2553 1204 2497 \n",
"Q 953 2441 691 2322 \n",
"L 691 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 52.160938 126.499219 \n",
"L 247.460938 126.499219 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use xlink:href=\"#m144608ae0a\" x=\"52.160938\" y=\"126.499219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −1.0 -->\n",
" <g transform=\"translate(20.878125 130.298438) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 52.160938 103.399219 \n",
"L 247.460938 103.399219 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <g>\n",
" <use xlink:href=\"#m144608ae0a\" x=\"52.160938\" y=\"103.399219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −0.5 -->\n",
" <g transform=\"translate(20.878125 107.198437) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 52.160938 80.299219 \n",
"L 247.460938 80.299219 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m144608ae0a\" x=\"52.160938\" y=\"80.299219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0.0 -->\n",
" <g transform=\"translate(29.257812 84.098438) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 52.160938 57.199219 \n",
"L 247.460938 57.199219 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m144608ae0a\" x=\"52.160938\" y=\"57.199219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 0.5 -->\n",
" <g transform=\"translate(29.257812 60.998437) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_6\">\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 52.160938 34.099219 \n",
"L 247.460938 34.099219 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_18\">\n",
" <g>\n",
" <use xlink:href=\"#m144608ae0a\" x=\"52.160938\" y=\"34.099219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 1.0 -->\n",
" <g transform=\"translate(29.257812 37.898438) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_7\">\n",
" <g id=\"line2d_19\">\n",
" <path d=\"M 52.160938 10.999219 \n",
"L 247.460938 10.999219 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_20\">\n",
" <g>\n",
" <use xlink:href=\"#m144608ae0a\" x=\"52.160938\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- 1.5 -->\n",
" <g transform=\"translate(29.257812 14.798438) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_12\">\n",
" <!-- f(x) -->\n",
" <g transform=\"translate(14.798438 88.920313) rotate(-90) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
"L 2375 4384 \n",
"L 1825 4384 \n",
"Q 1516 4384 1395 4259 \n",
"Q 1275 4134 1275 3809 \n",
"L 1275 3500 \n",
"L 2222 3500 \n",
"L 2222 3053 \n",
"L 1275 3053 \n",
"L 1275 0 \n",
"L 697 0 \n",
"L 697 3053 \n",
"L 147 3053 \n",
"L 147 3500 \n",
"L 697 3500 \n",
"L 697 3744 \n",
"Q 697 4328 969 4595 \n",
"Q 1241 4863 1831 4863 \n",
"L 2375 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
"Q 1566 4138 1362 3434 \n",
"Q 1159 2731 1159 2009 \n",
"Q 1159 1288 1364 580 \n",
"Q 1569 -128 1984 -844 \n",
"L 1484 -844 \n",
"Q 1016 -109 783 600 \n",
"Q 550 1309 550 2009 \n",
"Q 550 2706 781 3412 \n",
"Q 1013 4119 1484 4856 \n",
"L 1984 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
"L 1013 4856 \n",
"Q 1481 4119 1714 3412 \n",
"Q 1947 2706 1947 2009 \n",
"Q 1947 1309 1714 600 \n",
"Q 1481 -109 1013 -844 \n",
"L 513 -844 \n",
"Q 928 -128 1133 580 \n",
"Q 1338 1288 1338 2009 \n",
"Q 1338 2731 1133 3434 \n",
"Q 928 4138 513 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-66\"/>\n",
" <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
" <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
" <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_21\">\n",
" <path d=\"M 61.03821 80.299215 \n",
"L 69.51968 93.952246 \n",
"L 74.043133 100.809619 \n",
"L 77.718434 106.003092 \n",
"L 81.111026 110.413936 \n",
"L 84.220895 114.078152 \n",
"L 87.048056 117.056013 \n",
"L 89.592495 119.422848 \n",
"L 92.136934 121.472995 \n",
"L 94.398657 123.016012 \n",
"L 96.660387 124.285792 \n",
"L 98.639395 125.166274 \n",
"L 100.618407 125.826996 \n",
"L 102.597418 126.264724 \n",
"L 104.576426 126.477313 \n",
"L 106.555437 126.46372 \n",
"L 108.534445 126.224016 \n",
"L 110.513457 125.75937 \n",
"L 112.492465 125.072063 \n",
"L 114.471476 124.165456 \n",
"L 116.733199 122.866553 \n",
"L 118.994926 121.295363 \n",
"L 121.256653 119.461937 \n",
"L 123.801095 117.100546 \n",
"L 126.345535 114.441265 \n",
"L 129.172691 111.16339 \n",
"L 132.282566 107.202897 \n",
"L 135.675153 102.513228 \n",
"L 139.633173 96.643135 \n",
"L 144.439339 89.09675 \n",
"L 162.250429 60.687416 \n",
"L 166.208449 55.042095 \n",
"L 169.601036 50.592682 \n",
"L 172.710909 46.887931 \n",
"L 175.538066 43.869353 \n",
"L 178.082507 41.463066 \n",
"L 180.626949 39.371139 \n",
"L 182.888676 37.789303 \n",
"L 185.150403 36.479387 \n",
"L 187.412126 35.449769 \n",
"L 189.391134 34.784006 \n",
"L 191.370142 34.341178 \n",
"L 193.349153 34.123449 \n",
"L 195.328165 34.131892 \n",
"L 197.307172 34.36646 \n",
"L 199.286184 34.826009 \n",
"L 201.265195 35.508285 \n",
"L 203.244206 36.409949 \n",
"L 205.50593 37.70334 \n",
"L 207.767657 39.269204 \n",
"L 210.02938 41.097516 \n",
"L 212.573819 43.453451 \n",
"L 215.118264 46.107647 \n",
"L 217.945419 49.380334 \n",
"L 221.055288 53.335751 \n",
"L 224.44788 58.020721 \n",
"L 228.405903 63.886503 \n",
"L 233.212065 71.429462 \n",
"L 238.583665 80.152052 \n",
"L 238.583665 80.152052 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_22\">\n",
" <path d=\"M 61.03821 131.748311 \n",
"L 238.583665 111.224903 \n",
"L 238.583665 111.224903 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"line2d_23\">\n",
" <path d=\"M 83.854551 188.155469 \n",
"L 199.606108 -1 \n",
"L 199.606108 -1 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"line2d_24\">\n",
" <path d=\"M 148.624127 -1 \n",
"L 238.583665 60.176678 \n",
"L 238.583665 60.176678 \n",
"\" clip-path=\"url(#pe3fdd4b9d1)\" style=\"fill: none; stroke-dasharray: 1.5,2.475; stroke-dashoffset: 0; stroke: #ff0000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 52.160938 149.599219 \n",
"L 52.160938 10.999219 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 247.460938 149.599219 \n",
"L 247.460938 10.999219 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 52.160938 149.599219 \n",
"L 247.460938 149.599219 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 52.160938 10.999219 \n",
"L 247.460938 10.999219 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"pe3fdd4b9d1\">\n",
" <rect x=\"52.160938\" y=\"10.999219\" width=\"195.3\" height=\"138.6\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 350x250 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Compute sin\n",
"xs = torch.arange(-torch.pi, torch.pi, 0.01)\n",
"plots = [torch.sin(xs)]\n",
"\n",
"# Compute some linear approximations. Use d(sin(x))/dx = cos(x)\n",
"for x0 in [-1.5, 0.0, 2.0]:\n",
" plots.append(torch.sin(torch.tensor(x0)) + (xs - x0) *\n",
" torch.cos(torch.tensor(x0)))\n",
"\n",
"d2l.plot(xs, plots, 'x', 'f(x)', ylim=[-1.5, 1.5])"
]
},
{
"cell_type": "markdown",
"id": "53f57614",
"metadata": {
"origin_pos": 18
},
"source": [
"### Higher Order Derivatives\n",
"\n",
"Let's now do something that may on the surface seem strange. Take a function $f$ and compute the derivative $\\frac{df}{dx}$. This gives us the rate of change of $f$ at any point.\n",
"\n",
"However, the derivative, $\\frac{df}{dx}$, can be viewed as a function itself, so nothing stops us from computing the derivative of $\\frac{df}{dx}$ to get $\\frac{d^2f}{dx^2} = \\frac{df}{dx}\\left(\\frac{df}{dx}\\right)$. We will call this the second derivative of $f$. This function is the rate of change of the rate of change of $f$, or in other words, how the rate of change is changing. We may apply the derivative any number of times to obtain what is called the $n$-th derivative. To keep the notation clean, we will denote the $n$-th derivative as\n",
"\n",
"$$\n",
"f^{(n)}(x) = \\frac{d^{n}f}{dx^{n}} = \\left(\\frac{d}{dx}\\right)^{n} f.\n",
"$$\n",
"\n",
"Let's try to understand *why* this is a useful notion. Below, we visualize $f^{(2)}(x)$, $f^{(1)}(x)$, and $f(x)$.\n",
"\n",
"First, consider the case that the second derivative $f^{(2)}(x)$ is a positive constant. This means that the slope of the first derivative is positive. As a result, the first derivative $f^{(1)}(x)$ may start out negative, becomes zero at a point, and then becomes positive in the end. This tells us the slope of our original function $f$ and therefore, the function $f$ itself decreases, flattens out, then increases. In other words, the function $f$ curves up, and has a single minimum as is shown in :numref:`fig_positive-second`.\n",
"\n",
"![If we assume the second derivative is a positive constant, then the fist derivative in increasing, which implies the function itself has a minimum.](../img/posSecDer.svg)\n",
":label:`fig_positive-second`\n",
"\n",
"\n",
"Second, if the second derivative is a negative constant, that means that the first derivative is decreasing. This implies the first derivative may start out positive, becomes zero at a point, and then becomes negative. Hence, the function $f$ itself increases, flattens out, then decreases. In other words, the function $f$ curves down, and has a single maximum as is shown in :numref:`fig_negative-second`.\n",
"\n",
"![If we assume the second derivative is a negative constant, then the fist derivative in decreasing, which implies the function itself has a maximum.](../img/negSecDer.svg)\n",
":label:`fig_negative-second`\n",
"\n",
"\n",
"Third, if the second derivative is a always zero, then the first derivative will never change---it is constant! This means that $f$ increases (or decreases) at a fixed rate, and $f$ is itself a straight line as is shown in :numref:`fig_zero-second`.\n",
"\n",
"![If we assume the second derivative is zero, then the fist derivative is constant, which implies the function itself is a straight line.](../img/zeroSecDer.svg)\n",
":label:`fig_zero-second`\n",
"\n",
"To summarize, the second derivative can be interpreted as describing the way that the function $f$ curves. A positive second derivative leads to a upwards curve, while a negative second derivative means that $f$ curves downwards, and a zero second derivative means that $f$ does not curve at all.\n",
"\n",
"Let's take this one step further. Consider the function $g(x) = ax^{2}+ bx + c$. We can then compute that\n",
"\n",
"$$\n",
"\\begin{aligned}\n",
"\\frac{dg}{dx}(x) & = 2ax + b \\\\\n",
"\\frac{d^2g}{dx^2}(x) & = 2a.\n",
"\\end{aligned}\n",
"$$\n",
"\n",
"If we have some original function $f(x)$ in mind, we may compute the first two derivatives and find the values for $a, b$, and $c$ that make them match this computation. Similarly to the previous section where we saw that the first derivative gave the best approximation with a straight line, this construction provides the best approximation by a quadratic. Let's visualize this for $f(x) = \\sin(x)$.\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "e74b3a75",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T19:43:04.344302Z",
"iopub.status.busy": "2023-08-18T19:43:04.343971Z",
"iopub.status.idle": "2023-08-18T19:43:04.675314Z",
"shell.execute_reply": "2023-08-18T19:43:04.674049Z"
},
"origin_pos": 20,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"254.660937pt\" height=\"187.155469pt\" viewBox=\"0 0 254.660937 187.155469\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T19:43:04.636642</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 187.155469 \n",
"L 254.660937 187.155469 \n",
"L 254.660937 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 52.160938 149.599219 \n",
"L 247.460938 149.599219 \n",
"L 247.460938 10.999219 \n",
"L 52.160938 10.999219 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 93.312829 149.599219 \n",
"L 93.312829 10.999219 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"m8c51c2ae09\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m8c51c2ae09\" x=\"93.312829\" y=\"149.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- −2 -->\n",
" <g transform=\"translate(85.941735 164.197656) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
"L 4684 2272 \n",
"L 4684 1741 \n",
"L 678 1741 \n",
"L 678 2272 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 149.855967 149.599219 \n",
"L 149.855967 10.999219 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#m8c51c2ae09\" x=\"149.855967\" y=\"149.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(146.674717 164.197656) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 206.399106 149.599219 \n",
"L 206.399106 10.999219 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#m8c51c2ae09\" x=\"206.399106\" y=\"149.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(203.217856 164.197656) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- x -->\n",
" <g transform=\"translate(146.851563 177.875781) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 52.160938 149.599219 \n",
"L 247.460938 149.599219 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <defs>\n",
" <path id=\"m24363d02b5\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m24363d02b5\" x=\"52.160938\" y=\"149.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- −1.5 -->\n",
" <g transform=\"translate(20.878125 153.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
"L 1344 794 \n",
"L 1344 0 \n",
"L 684 0 \n",
"L 684 794 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
"L 3169 4666 \n",
"L 3169 4134 \n",
"L 1269 4134 \n",
"L 1269 2991 \n",
"Q 1406 3038 1543 3061 \n",
"Q 1681 3084 1819 3084 \n",
"Q 2600 3084 3056 2656 \n",
"Q 3513 2228 3513 1497 \n",
"Q 3513 744 3044 326 \n",
"Q 2575 -91 1722 -91 \n",
"Q 1428 -91 1123 -41 \n",
"Q 819 9 494 109 \n",
"L 494 744 \n",
"Q 775 591 1075 516 \n",
"Q 1375 441 1709 441 \n",
"Q 2250 441 2565 725 \n",
"Q 2881 1009 2881 1497 \n",
"Q 2881 1984 2565 2268 \n",
"Q 2250 2553 1709 2553 \n",
"Q 1456 2553 1204 2497 \n",
"Q 953 2441 691 2322 \n",
"L 691 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 52.160938 126.499219 \n",
"L 247.460938 126.499219 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use xlink:href=\"#m24363d02b5\" x=\"52.160938\" y=\"126.499219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- −1.0 -->\n",
" <g transform=\"translate(20.878125 130.298438) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 52.160938 103.399219 \n",
"L 247.460938 103.399219 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <g>\n",
" <use xlink:href=\"#m24363d02b5\" x=\"52.160938\" y=\"103.399219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- −0.5 -->\n",
" <g transform=\"translate(20.878125 107.198437) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-2212\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"179.199219\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 52.160938 80.299219 \n",
"L 247.460938 80.299219 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m24363d02b5\" x=\"52.160938\" y=\"80.299219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 0.0 -->\n",
" <g transform=\"translate(29.257812 84.098438) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 52.160938 57.199219 \n",
"L 247.460938 57.199219 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m24363d02b5\" x=\"52.160938\" y=\"57.199219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 0.5 -->\n",
" <g transform=\"translate(29.257812 60.998437) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_6\">\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 52.160938 34.099219 \n",
"L 247.460938 34.099219 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_18\">\n",
" <g>\n",
" <use xlink:href=\"#m24363d02b5\" x=\"52.160938\" y=\"34.099219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- 1.0 -->\n",
" <g transform=\"translate(29.257812 37.898438) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_7\">\n",
" <g id=\"line2d_19\">\n",
" <path d=\"M 52.160938 10.999219 \n",
"L 247.460938 10.999219 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_20\">\n",
" <g>\n",
" <use xlink:href=\"#m24363d02b5\" x=\"52.160938\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- 1.5 -->\n",
" <g transform=\"translate(29.257812 14.798438) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_12\">\n",
" <!-- f(x) -->\n",
" <g transform=\"translate(14.798438 88.920313) rotate(-90) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
"L 2375 4384 \n",
"L 1825 4384 \n",
"Q 1516 4384 1395 4259 \n",
"Q 1275 4134 1275 3809 \n",
"L 1275 3500 \n",
"L 2222 3500 \n",
"L 2222 3053 \n",
"L 1275 3053 \n",
"L 1275 0 \n",
"L 697 0 \n",
"L 697 3053 \n",
"L 147 3053 \n",
"L 147 3500 \n",
"L 697 3500 \n",
"L 697 3744 \n",
"Q 697 4328 969 4595 \n",
"Q 1241 4863 1831 4863 \n",
"L 2375 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
"Q 1566 4138 1362 3434 \n",
"Q 1159 2731 1159 2009 \n",
"Q 1159 1288 1364 580 \n",
"Q 1569 -128 1984 -844 \n",
"L 1484 -844 \n",
"Q 1016 -109 783 600 \n",
"Q 550 1309 550 2009 \n",
"Q 550 2706 781 3412 \n",
"Q 1013 4119 1484 4856 \n",
"L 1984 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
"L 1013 4856 \n",
"Q 1481 4119 1714 3412 \n",
"Q 1947 2706 1947 2009 \n",
"Q 1947 1309 1714 600 \n",
"Q 1481 -109 1013 -844 \n",
"L 513 -844 \n",
"Q 928 -128 1133 580 \n",
"Q 1338 1288 1338 2009 \n",
"Q 1338 2731 1133 3434 \n",
"Q 928 4138 513 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-66\"/>\n",
" <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
" <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
" <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_21\">\n",
" <path d=\"M 61.03821 80.299215 \n",
"L 69.51968 93.952246 \n",
"L 74.043133 100.809619 \n",
"L 77.718434 106.003092 \n",
"L 81.111026 110.413936 \n",
"L 84.220895 114.078152 \n",
"L 87.048056 117.056013 \n",
"L 89.592495 119.422848 \n",
"L 92.136934 121.472995 \n",
"L 94.398657 123.016012 \n",
"L 96.660387 124.285792 \n",
"L 98.639395 125.166274 \n",
"L 100.618407 125.826996 \n",
"L 102.597418 126.264724 \n",
"L 104.576426 126.477313 \n",
"L 106.555437 126.46372 \n",
"L 108.534445 126.224016 \n",
"L 110.513457 125.75937 \n",
"L 112.492465 125.072063 \n",
"L 114.471476 124.165456 \n",
"L 116.733199 122.866553 \n",
"L 118.994926 121.295363 \n",
"L 121.256653 119.461937 \n",
"L 123.801095 117.100546 \n",
"L 126.345535 114.441265 \n",
"L 129.172691 111.16339 \n",
"L 132.282566 107.202897 \n",
"L 135.675153 102.513228 \n",
"L 139.633173 96.643135 \n",
"L 144.439339 89.09675 \n",
"L 162.250429 60.687416 \n",
"L 166.208449 55.042095 \n",
"L 169.601036 50.592682 \n",
"L 172.710909 46.887931 \n",
"L 175.538066 43.869353 \n",
"L 178.082507 41.463066 \n",
"L 180.626949 39.371139 \n",
"L 182.888676 37.789303 \n",
"L 185.150403 36.479387 \n",
"L 187.412126 35.449769 \n",
"L 189.391134 34.784006 \n",
"L 191.370142 34.341178 \n",
"L 193.349153 34.123449 \n",
"L 195.328165 34.131892 \n",
"L 197.307172 34.36646 \n",
"L 199.286184 34.826009 \n",
"L 201.265195 35.508285 \n",
"L 203.244206 36.409949 \n",
"L 205.50593 37.70334 \n",
"L 207.767657 39.269204 \n",
"L 210.02938 41.097516 \n",
"L 212.573819 43.453451 \n",
"L 215.118264 46.107647 \n",
"L 217.945419 49.380334 \n",
"L 221.055288 53.335751 \n",
"L 224.44788 58.020721 \n",
"L 228.405903 63.886503 \n",
"L 233.212065 71.429462 \n",
"L 238.583665 80.152052 \n",
"L 238.583665 80.152052 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_22\">\n",
" <path d=\"M 61.03821 69.653749 \n",
"L 64.430795 78.007957 \n",
"L 67.540672 85.083037 \n",
"L 70.650541 91.600477 \n",
"L 73.477702 97.041556 \n",
"L 76.304857 102.021773 \n",
"L 79.132018 106.541161 \n",
"L 81.676456 110.21458 \n",
"L 84.220895 113.514721 \n",
"L 86.765341 116.441583 \n",
"L 89.027064 118.72986 \n",
"L 91.288788 120.7232 \n",
"L 93.550511 122.421596 \n",
"L 95.812241 123.825062 \n",
"L 97.791249 124.811146 \n",
"L 99.77026 125.571419 \n",
"L 101.749272 126.105881 \n",
"L 103.728283 126.41453 \n",
"L 105.707288 126.497363 \n",
"L 107.686299 126.354386 \n",
"L 109.665307 125.985592 \n",
"L 111.644318 125.390987 \n",
"L 113.62333 124.570569 \n",
"L 115.602338 123.52434 \n",
"L 117.864064 122.052141 \n",
"L 120.125791 120.285004 \n",
"L 122.387516 118.222927 \n",
"L 124.649241 115.865915 \n",
"L 127.193682 112.861728 \n",
"L 129.738122 109.484259 \n",
"L 132.282566 105.733504 \n",
"L 134.827007 101.60947 \n",
"L 137.654162 96.589409 \n",
"L 140.48132 91.108511 \n",
"L 143.308477 85.166761 \n",
"L 146.418349 78.098566 \n",
"L 149.528222 70.472751 \n",
"L 152.920811 61.517719 \n",
"L 156.313399 51.899073 \n",
"L 159.705987 41.616807 \n",
"L 163.381291 29.728822 \n",
"L 167.056595 17.062017 \n",
"L 170.731899 3.616401 \n",
"L 171.946816 -1 \n",
"L 171.946816 -1 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"line2d_23\">\n",
" <path d=\"M 83.854551 188.155469 \n",
"L 199.606108 -1 \n",
"L 199.606108 -1 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"line2d_24\">\n",
" <path d=\"M 116.843555 188.155469 \n",
"L 120.973937 171.970603 \n",
"L 124.931957 157.302945 \n",
"L 128.889976 143.458677 \n",
"L 132.847997 130.437764 \n",
"L 136.5233 119.084194 \n",
"L 140.198605 108.440607 \n",
"L 143.873908 98.506956 \n",
"L 147.266497 89.967586 \n",
"L 150.659085 82.033152 \n",
"L 153.768958 75.291338 \n",
"L 156.87883 69.057835 \n",
"L 159.988703 63.332661 \n",
"L 162.81586 58.569047 \n",
"L 165.643016 54.225535 \n",
"L 168.470174 50.302119 \n",
"L 171.014614 47.130219 \n",
"L 173.559057 44.298603 \n",
"L 176.103497 41.807263 \n",
"L 178.365222 39.878399 \n",
"L 180.626949 38.218404 \n",
"L 182.888676 36.827267 \n",
"L 185.150403 35.704987 \n",
"L 187.412126 34.851569 \n",
"L 189.391134 34.325386 \n",
"L 191.370142 34.005041 \n",
"L 193.349153 33.890546 \n",
"L 195.328165 33.981893 \n",
"L 197.307172 34.279093 \n",
"L 199.286184 34.782139 \n",
"L 201.265195 35.491033 \n",
"L 203.526919 36.553251 \n",
"L 205.788645 37.884329 \n",
"L 208.050372 39.484271 \n",
"L 210.312095 41.353073 \n",
"L 212.573819 43.490734 \n",
"L 215.118264 46.216984 \n",
"L 217.662703 49.283501 \n",
"L 220.207142 52.690298 \n",
"L 223.034303 56.874724 \n",
"L 225.861457 61.479238 \n",
"L 228.688618 66.503858 \n",
"L 231.798488 72.516139 \n",
"L 234.908358 79.036732 \n",
"L 238.018234 86.065656 \n",
"L 238.583665 87.39825 \n",
"L 238.583665 87.39825 \n",
"\" clip-path=\"url(#pfe28aa18f4)\" style=\"fill: none; stroke-dasharray: 1.5,2.475; stroke-dashoffset: 0; stroke: #ff0000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 52.160938 149.599219 \n",
"L 52.160938 10.999219 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 247.460938 149.599219 \n",
"L 247.460938 10.999219 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 52.160938 149.599219 \n",
"L 247.460938 149.599219 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 52.160938 10.999219 \n",
"L 247.460938 10.999219 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"pfe28aa18f4\">\n",
" <rect x=\"52.160938\" y=\"10.999219\" width=\"195.3\" height=\"138.6\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 350x250 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Compute sin\n",
"xs = torch.arange(-torch.pi, torch.pi, 0.01)\n",
"plots = [torch.sin(xs)]\n",
"\n",
"# Compute some quadratic approximations. Use d(sin(x)) / dx = cos(x)\n",
"for x0 in [-1.5, 0.0, 2.0]:\n",
" plots.append(torch.sin(torch.tensor(x0)) + (xs - x0) *\n",
" torch.cos(torch.tensor(x0)) - (xs - x0)**2 *\n",
" torch.sin(torch.tensor(x0)) / 2)\n",
"\n",
"d2l.plot(xs, plots, 'x', 'f(x)', ylim=[-1.5, 1.5])"
]
},
{
"cell_type": "markdown",
"id": "752c3096",
"metadata": {
"origin_pos": 22
},
"source": [
"We will extend this idea to the idea of a *Taylor series* in the next section.\n",
"\n",
"### Taylor Series\n",
"\n",
"\n",
"The *Taylor series* provides a method to approximate the function $f(x)$ if we are given values for the first $n$ derivatives at a point $x_0$, i.e., $\\left\\{ f(x_0), f^{(1)}(x_0), f^{(2)}(x_0), \\ldots, f^{(n)}(x_0) \\right\\}$. The idea will be to find a degree $n$ polynomial that matches all the given derivatives at $x_0$.\n",
"\n",
"We saw the case of $n=2$ in the previous section and a little algebra shows this is\n",
"\n",
"$$\n",
"f(x) \\approx \\frac{1}{2}\\frac{d^2f}{dx^2}(x_0)(x-x_0)^{2}+ \\frac{df}{dx}(x_0)(x-x_0) + f(x_0).\n",
"$$\n",
"\n",
"As we can see above, the denominator of $2$ is there to cancel out the $2$ we get when we take two derivatives of $x^2$, while the other terms are all zero. Same logic applies for the first derivative and the value itself.\n",
"\n",
"If we push the logic further to $n=3$, we will conclude that\n",
"\n",
"$$\n",
"f(x) \\approx \\frac{\\frac{d^3f}{dx^3}(x_0)}{6}(x-x_0)^3 + \\frac{\\frac{d^2f}{dx^2}(x_0)}{2}(x-x_0)^{2}+ \\frac{df}{dx}(x_0)(x-x_0) + f(x_0).\n",
"$$\n",
"\n",
"where the $6 = 3 \\times 2 = 3!$ comes from the constant we get in front if we take three derivatives of $x^3$.\n",
"\n",
"\n",
"Furthermore, we can get a degree $n$ polynomial by\n",
"\n",
"$$\n",
"P_n(x) = \\sum_{i = 0}^{n} \\frac{f^{(i)}(x_0)}{i!}(x-x_0)^{i}.\n",
"$$\n",
"\n",
"where the notation\n",
"\n",
"$$\n",
"f^{(n)}(x) = \\frac{d^{n}f}{dx^{n}} = \\left(\\frac{d}{dx}\\right)^{n} f.\n",
"$$\n",
"\n",
"\n",
"Indeed, $P_n(x)$ can be viewed as the best $n$-th degree polynomial approximation to our function $f(x)$.\n",
"\n",
"While we are not going to dive all the way into the error of the above approximations, it is worth mentioning the infinite limit. In this case, for well behaved functions (known as real analytic functions) like $\\cos(x)$ or $e^{x}$, we can write out the infinite number of terms and approximate the exactly same function\n",
"\n",
"$$\n",
"f(x) = \\sum_{n = 0}^\\infty \\frac{f^{(n)}(x_0)}{n!}(x-x_0)^{n}.\n",
"$$\n",
"\n",
"Take $f(x) = e^{x}$ as am example. Since $e^{x}$ is its own derivative, we know that $f^{(n)}(x) = e^{x}$. Therefore, $e^{x}$ can be reconstructed by taking the Taylor series at $x_0 = 0$, i.e.,\n",
"\n",
"$$\n",
"e^{x} = \\sum_{n = 0}^\\infty \\frac{x^{n}}{n!} = 1 + x + \\frac{x^2}{2} + \\frac{x^3}{6} + \\cdots.\n",
"$$\n",
"\n",
"Let's see how this works in code and observe how increasing the degree of the Taylor approximation brings us closer to the desired function $e^x$.\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "63cd6f06",
"metadata": {
"execution": {
"iopub.execute_input": "2023-08-18T19:43:04.679251Z",
"iopub.status.busy": "2023-08-18T19:43:04.678467Z",
"iopub.status.idle": "2023-08-18T19:43:04.978071Z",
"shell.execute_reply": "2023-08-18T19:43:04.976879Z"
},
"origin_pos": 24,
"tab": [
"pytorch"
]
},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"243.103125pt\" height=\"183.35625pt\" viewBox=\"0 0 243.103125 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
" <metadata>\n",
" <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
" <cc:Work>\n",
" <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
" <dc:date>2023-08-18T19:43:04.919828</dc:date>\n",
" <dc:format>image/svg+xml</dc:format>\n",
" <dc:creator>\n",
" <cc:Agent>\n",
" <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
" </cc:Agent>\n",
" </dc:creator>\n",
" </cc:Work>\n",
" </rdf:RDF>\n",
" </metadata>\n",
" <defs>\n",
" <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 183.35625 \n",
"L 243.103125 183.35625 \n",
"L 243.103125 0 \n",
"L 0 0 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 40.603125 145.8 \n",
"L 235.903125 145.8 \n",
"L 235.903125 7.2 \n",
"L 40.603125 7.2 \n",
"z\n",
"\" style=\"fill: #ffffff\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <path d=\"M 49.480398 145.8 \n",
"L 49.480398 7.2 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_2\">\n",
" <defs>\n",
" <path id=\"mbf72cae969\" d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#mbf72cae969\" x=\"49.480398\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 0 -->\n",
" <g transform=\"translate(46.299148 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
"Q 1547 4250 1301 3770 \n",
"Q 1056 3291 1056 2328 \n",
"Q 1056 1369 1301 889 \n",
"Q 1547 409 2034 409 \n",
"Q 2525 409 2770 889 \n",
"Q 3016 1369 3016 2328 \n",
"Q 3016 3291 2770 3770 \n",
"Q 2525 4250 2034 4250 \n",
"z\n",
"M 2034 4750 \n",
"Q 2819 4750 3233 4129 \n",
"Q 3647 3509 3647 2328 \n",
"Q 3647 1150 3233 529 \n",
"Q 2819 -91 2034 -91 \n",
"Q 1250 -91 836 529 \n",
"Q 422 1150 422 2328 \n",
"Q 422 3509 836 4129 \n",
"Q 1250 4750 2034 4750 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_3\">\n",
" <path d=\"M 108.860148 145.8 \n",
"L 108.860148 7.2 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use xlink:href=\"#mbf72cae969\" x=\"108.860148\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 1 -->\n",
" <g transform=\"translate(105.678898 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
"L 1825 531 \n",
"L 1825 4091 \n",
"L 703 3866 \n",
"L 703 4441 \n",
"L 1819 4666 \n",
"L 2450 4666 \n",
"L 2450 531 \n",
"L 3481 531 \n",
"L 3481 0 \n",
"L 794 0 \n",
"L 794 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_5\">\n",
" <path d=\"M 168.239899 145.8 \n",
"L 168.239899 7.2 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use xlink:href=\"#mbf72cae969\" x=\"168.239899\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 2 -->\n",
" <g transform=\"translate(165.058649 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
"L 3431 531 \n",
"L 3431 0 \n",
"L 469 0 \n",
"L 469 531 \n",
"Q 828 903 1448 1529 \n",
"Q 2069 2156 2228 2338 \n",
"Q 2531 2678 2651 2914 \n",
"Q 2772 3150 2772 3378 \n",
"Q 2772 3750 2511 3984 \n",
"Q 2250 4219 1831 4219 \n",
"Q 1534 4219 1204 4116 \n",
"Q 875 4013 500 3803 \n",
"L 500 4441 \n",
"Q 881 4594 1212 4672 \n",
"Q 1544 4750 1819 4750 \n",
"Q 2544 4750 2975 4387 \n",
"Q 3406 4025 3406 3419 \n",
"Q 3406 3131 3298 2873 \n",
"Q 3191 2616 2906 2266 \n",
"Q 2828 2175 2409 1742 \n",
"Q 1991 1309 1228 531 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_7\">\n",
" <path d=\"M 227.619649 145.8 \n",
"L 227.619649 7.2 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_8\">\n",
" <g>\n",
" <use xlink:href=\"#mbf72cae969\" x=\"227.619649\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 3 -->\n",
" <g transform=\"translate(224.438399 160.398438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
"Q 3050 2419 3304 2112 \n",
"Q 3559 1806 3559 1356 \n",
"Q 3559 666 3084 287 \n",
"Q 2609 -91 1734 -91 \n",
"Q 1441 -91 1130 -33 \n",
"Q 819 25 488 141 \n",
"L 488 750 \n",
"Q 750 597 1062 519 \n",
"Q 1375 441 1716 441 \n",
"Q 2309 441 2620 675 \n",
"Q 2931 909 2931 1356 \n",
"Q 2931 1769 2642 2001 \n",
"Q 2353 2234 1838 2234 \n",
"L 1294 2234 \n",
"L 1294 2753 \n",
"L 1863 2753 \n",
"Q 2328 2753 2575 2939 \n",
"Q 2822 3125 2822 3475 \n",
"Q 2822 3834 2567 4026 \n",
"Q 2313 4219 1838 4219 \n",
"Q 1578 4219 1281 4162 \n",
"Q 984 4106 628 3988 \n",
"L 628 4550 \n",
"Q 988 4650 1302 4700 \n",
"Q 1616 4750 1894 4750 \n",
"Q 2613 4750 3031 4423 \n",
"Q 3450 4097 3450 3541 \n",
"Q 3450 3153 3228 2886 \n",
"Q 3006 2619 2597 2516 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-33\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- x -->\n",
" <g transform=\"translate(135.29375 174.076563) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
"L 2247 1797 \n",
"L 3578 0 \n",
"L 2900 0 \n",
"L 1881 1375 \n",
"L 863 0 \n",
"L 184 0 \n",
"L 1544 1831 \n",
"L 300 3500 \n",
"L 978 3500 \n",
"L 1906 2253 \n",
"L 2834 3500 \n",
"L 3513 3500 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-78\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_9\">\n",
" <path d=\"M 40.603125 112.813118 \n",
"L 235.903125 112.813118 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_10\">\n",
" <defs>\n",
" <path id=\"m54d444cb1b\" d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </defs>\n",
" <g>\n",
" <use xlink:href=\"#m54d444cb1b\" x=\"40.603125\" y=\"112.813118\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- 5 -->\n",
" <g transform=\"translate(27.240625 116.612337) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
"L 3169 4666 \n",
"L 3169 4134 \n",
"L 1269 4134 \n",
"L 1269 2991 \n",
"Q 1406 3038 1543 3061 \n",
"Q 1681 3084 1819 3084 \n",
"Q 2600 3084 3056 2656 \n",
"Q 3513 2228 3513 1497 \n",
"Q 3513 744 3044 326 \n",
"Q 2575 -91 1722 -91 \n",
"Q 1428 -91 1123 -41 \n",
"Q 819 9 494 109 \n",
"L 494 744 \n",
"Q 775 591 1075 516 \n",
"Q 1375 441 1709 441 \n",
"Q 2250 441 2565 725 \n",
"Q 2881 1009 2881 1497 \n",
"Q 2881 1984 2565 2268 \n",
"Q 2250 2553 1709 2553 \n",
"Q 1456 2553 1204 2497 \n",
"Q 953 2441 691 2322 \n",
"L 691 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-35\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_11\">\n",
" <path d=\"M 40.603125 79.454515 \n",
"L 235.903125 79.454515 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_12\">\n",
" <g>\n",
" <use xlink:href=\"#m54d444cb1b\" x=\"40.603125\" y=\"79.454515\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- 10 -->\n",
" <g transform=\"translate(20.878125 83.253734) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_13\">\n",
" <path d=\"M 40.603125 46.095912 \n",
"L 235.903125 46.095912 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use xlink:href=\"#m54d444cb1b\" x=\"40.603125\" y=\"46.095912\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- 15 -->\n",
" <g transform=\"translate(20.878125 49.895131) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"63.623047\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_15\">\n",
" <path d=\"M 40.603125 12.737309 \n",
"L 235.903125 12.737309 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use xlink:href=\"#m54d444cb1b\" x=\"40.603125\" y=\"12.737309\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- 20 -->\n",
" <g transform=\"translate(20.878125 16.536528) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_10\">\n",
" <!-- f(x) -->\n",
" <g transform=\"translate(14.798437 85.121094) rotate(-90) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
"L 2375 4384 \n",
"L 1825 4384 \n",
"Q 1516 4384 1395 4259 \n",
"Q 1275 4134 1275 3809 \n",
"L 1275 3500 \n",
"L 2222 3500 \n",
"L 2222 3053 \n",
"L 1275 3053 \n",
"L 1275 0 \n",
"L 697 0 \n",
"L 697 3053 \n",
"L 147 3053 \n",
"L 147 3500 \n",
"L 697 3500 \n",
"L 697 3744 \n",
"Q 697 4328 969 4595 \n",
"Q 1241 4863 1831 4863 \n",
"L 2375 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
"Q 1566 4138 1362 3434 \n",
"Q 1159 2731 1159 2009 \n",
"Q 1159 1288 1364 580 \n",
"Q 1569 -128 1984 -844 \n",
"L 1484 -844 \n",
"Q 1016 -109 783 600 \n",
"Q 550 1309 550 2009 \n",
"Q 550 2706 781 3412 \n",
"Q 1013 4119 1484 4856 \n",
"L 1984 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
"L 1013 4856 \n",
"Q 1481 4119 1714 3412 \n",
"Q 1947 2706 1947 2009 \n",
"Q 1947 1309 1714 600 \n",
"Q 1481 -109 1013 -844 \n",
"L 513 -844 \n",
"Q 928 -128 1133 580 \n",
"Q 1338 1288 1338 2009 \n",
"Q 1338 2731 1133 3434 \n",
"Q 928 4138 513 4856 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-66\"/>\n",
" <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
" <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
" <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_17\">\n",
" <path d=\"M 49.480398 139.5 \n",
"L 60.168752 138.184221 \n",
"L 69.669513 136.798303 \n",
"L 78.576474 135.281363 \n",
"L 86.88964 133.644828 \n",
"L 94.609011 131.905738 \n",
"L 101.734578 130.086872 \n",
"L 108.860148 128.036104 \n",
"L 115.391922 125.927329 \n",
"L 121.923695 123.573351 \n",
"L 127.861664 121.196661 \n",
"L 133.799641 118.57001 \n",
"L 139.143827 115.970633 \n",
"L 144.488 113.126472 \n",
"L 149.832172 110.014462 \n",
"L 154.582555 107.003027 \n",
"L 159.332938 103.740783 \n",
"L 164.083313 100.206837 \n",
"L 168.833696 96.378552 \n",
"L 173.584071 92.231432 \n",
"L 177.740664 88.320306 \n",
"L 181.897242 84.125609 \n",
"L 186.053821 79.626765 \n",
"L 190.2104 74.80172 \n",
"L 194.366992 69.626793 \n",
"L 198.523571 64.076668 \n",
"L 202.680149 58.124111 \n",
"L 206.836742 51.739919 \n",
"L 210.993321 44.89285 \n",
"L 215.149899 37.54931 \n",
"L 218.712681 30.832478 \n",
"L 222.275477 23.70028 \n",
"L 225.838258 16.127074 \n",
"L 227.025852 13.5 \n",
"L 227.025852 13.5 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"line2d_18\">\n",
" <path d=\"M 49.480398 139.5 \n",
"L 227.025852 119.551555 \n",
"L 227.025852 119.551555 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"line2d_19\">\n",
" <path d=\"M 49.480398 139.5 \n",
"L 60.168752 138.191009 \n",
"L 70.857107 136.665853 \n",
"L 81.545461 134.924534 \n",
"L 92.23382 132.967051 \n",
"L 102.922172 130.793406 \n",
"L 113.610524 128.403595 \n",
"L 124.892687 125.646505 \n",
"L 136.174836 122.648567 \n",
"L 147.456985 119.409781 \n",
"L 158.739141 115.930144 \n",
"L 170.02129 112.209662 \n",
"L 181.303446 108.248325 \n",
"L 192.585602 104.046142 \n",
"L 203.867743 99.603114 \n",
"L 215.149899 94.919231 \n",
"L 226.432055 89.994497 \n",
"L 227.025852 89.728633 \n",
"L 227.025852 89.728633 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"line2d_20\">\n",
" <path d=\"M 49.480398 139.5 \n",
"L 60.168752 138.184222 \n",
"L 69.669513 136.798317 \n",
"L 78.576474 135.281501 \n",
"L 86.88964 133.645463 \n",
"L 94.609011 131.907738 \n",
"L 101.734578 130.091782 \n",
"L 108.860148 128.046878 \n",
"L 115.391922 125.947843 \n",
"L 121.923695 123.610165 \n",
"L 127.861664 121.256704 \n",
"L 133.799641 118.664626 \n",
"L 139.737617 115.811856 \n",
"L 145.081797 113.00192 \n",
"L 150.425969 109.94322 \n",
"L 155.770149 106.616168 \n",
"L 161.114328 103.000002 \n",
"L 165.864704 99.525188 \n",
"L 170.615086 95.788309 \n",
"L 175.365462 91.772297 \n",
"L 180.115852 87.459229 \n",
"L 184.866227 82.830345 \n",
"L 189.616603 77.86601 \n",
"L 194.366992 72.545658 \n",
"L 199.117368 66.847881 \n",
"L 203.273946 61.535108 \n",
"L 207.430539 55.900607 \n",
"L 211.587118 49.928218 \n",
"L 215.743696 43.601156 \n",
"L 219.900275 36.902063 \n",
"L 224.056868 29.812915 \n",
"L 227.025852 24.50023 \n",
"L 227.025852 24.50023 \n",
"\" clip-path=\"url(#pa4fea8ada3)\" style=\"fill: none; stroke-dasharray: 1.5,2.475; stroke-dashoffset: 0; stroke: #ff0000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 40.603125 145.8 \n",
"L 40.603125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 235.903125 145.8 \n",
"L 235.903125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 40.603125 145.8 \n",
"L 235.903125 145.8 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 40.603125 7.2 \n",
"L 235.903125 7.2 \n",
"\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"legend_1\">\n",
" <g id=\"patch_7\">\n",
" <path d=\"M 47.603125 73.9125 \n",
"L 192.159375 73.9125 \n",
"Q 194.159375 73.9125 194.159375 71.9125 \n",
"L 194.159375 14.2 \n",
"Q 194.159375 12.2 192.159375 12.2 \n",
"L 47.603125 12.2 \n",
"Q 45.603125 12.2 45.603125 14.2 \n",
"L 45.603125 71.9125 \n",
"Q 45.603125 73.9125 47.603125 73.9125 \n",
"z\n",
"\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
" </g>\n",
" <g id=\"line2d_21\">\n",
" <path d=\"M 49.603125 20.298438 \n",
"L 59.603125 20.298438 \n",
"L 69.603125 20.298438 \n",
"\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
" </g>\n",
" <g id=\"text_11\">\n",
" <!-- Exponential -->\n",
" <g transform=\"translate(77.603125 23.798438) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-45\" d=\"M 628 4666 \n",
"L 3578 4666 \n",
"L 3578 4134 \n",
"L 1259 4134 \n",
"L 1259 2753 \n",
"L 3481 2753 \n",
"L 3481 2222 \n",
"L 1259 2222 \n",
"L 1259 531 \n",
"L 3634 531 \n",
"L 3634 0 \n",
"L 628 0 \n",
"L 628 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
"L 1159 -1331 \n",
"L 581 -1331 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2969 \n",
"Q 1341 3281 1617 3432 \n",
"Q 1894 3584 2278 3584 \n",
"Q 2916 3584 3314 3078 \n",
"Q 3713 2572 3713 1747 \n",
"Q 3713 922 3314 415 \n",
"Q 2916 -91 2278 -91 \n",
"Q 1894 -91 1617 61 \n",
"Q 1341 213 1159 525 \n",
"z\n",
"M 3116 1747 \n",
"Q 3116 2381 2855 2742 \n",
"Q 2594 3103 2138 3103 \n",
"Q 1681 3103 1420 2742 \n",
"Q 1159 2381 1159 1747 \n",
"Q 1159 1113 1420 752 \n",
"Q 1681 391 2138 391 \n",
"Q 2594 391 2855 752 \n",
"Q 3116 1113 3116 1747 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
"Q 1497 3097 1228 2736 \n",
"Q 959 2375 959 1747 \n",
"Q 959 1119 1226 758 \n",
"Q 1494 397 1959 397 \n",
"Q 2419 397 2687 759 \n",
"Q 2956 1122 2956 1747 \n",
"Q 2956 2369 2687 2733 \n",
"Q 2419 3097 1959 3097 \n",
"z\n",
"M 1959 3584 \n",
"Q 2709 3584 3137 3096 \n",
"Q 3566 2609 3566 1747 \n",
"Q 3566 888 3137 398 \n",
"Q 2709 -91 1959 -91 \n",
"Q 1206 -91 779 398 \n",
"Q 353 888 353 1747 \n",
"Q 353 2609 779 3096 \n",
"Q 1206 3584 1959 3584 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
"L 3513 0 \n",
"L 2938 0 \n",
"L 2938 2094 \n",
"Q 2938 2591 2744 2837 \n",
"Q 2550 3084 2163 3084 \n",
"Q 1697 3084 1428 2787 \n",
"Q 1159 2491 1159 1978 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1366 3272 1645 3428 \n",
"Q 1925 3584 2291 3584 \n",
"Q 2894 3584 3203 3211 \n",
"Q 3513 2838 3513 2113 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
"L 3597 1613 \n",
"L 953 1613 \n",
"Q 991 1019 1311 708 \n",
"Q 1631 397 2203 397 \n",
"Q 2534 397 2845 478 \n",
"Q 3156 559 3463 722 \n",
"L 3463 178 \n",
"Q 3153 47 2828 -22 \n",
"Q 2503 -91 2169 -91 \n",
"Q 1331 -91 842 396 \n",
"Q 353 884 353 1716 \n",
"Q 353 2575 817 3079 \n",
"Q 1281 3584 2069 3584 \n",
"Q 2775 3584 3186 3129 \n",
"Q 3597 2675 3597 1894 \n",
"z\n",
"M 3022 2063 \n",
"Q 3016 2534 2758 2815 \n",
"Q 2500 3097 2075 3097 \n",
"Q 1594 3097 1305 2825 \n",
"Q 1016 2553 972 2059 \n",
"L 3022 2063 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
"L 1172 3500 \n",
"L 2356 3500 \n",
"L 2356 3053 \n",
"L 1172 3053 \n",
"L 1172 1153 \n",
"Q 1172 725 1289 603 \n",
"Q 1406 481 1766 481 \n",
"L 2356 481 \n",
"L 2356 0 \n",
"L 1766 0 \n",
"Q 1100 0 847 248 \n",
"Q 594 497 594 1153 \n",
"L 594 3053 \n",
"L 172 3053 \n",
"L 172 3500 \n",
"L 594 3500 \n",
"L 594 4494 \n",
"L 1172 4494 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
"L 1178 3500 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 3500 \n",
"z\n",
"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 4134 \n",
"L 603 4134 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
"Q 1497 1759 1228 1600 \n",
"Q 959 1441 959 1056 \n",
"Q 959 750 1161 570 \n",
"Q 1363 391 1709 391 \n",
"Q 2188 391 2477 730 \n",
"Q 2766 1069 2766 1631 \n",
"L 2766 1759 \n",
"L 2194 1759 \n",
"z\n",
"M 3341 1997 \n",
"L 3341 0 \n",
"L 2766 0 \n",
"L 2766 531 \n",
"Q 2569 213 2275 61 \n",
"Q 1981 -91 1556 -91 \n",
"Q 1019 -91 701 211 \n",
"Q 384 513 384 1019 \n",
"Q 384 1609 779 1909 \n",
"Q 1175 2209 1959 2209 \n",
"L 2766 2209 \n",
"L 2766 2266 \n",
"Q 2766 2663 2505 2880 \n",
"Q 2244 3097 1772 3097 \n",
"Q 1472 3097 1187 3025 \n",
"Q 903 2953 641 2809 \n",
"L 641 3341 \n",
"Q 956 3463 1253 3523 \n",
"Q 1550 3584 1831 3584 \n",
"Q 2591 3584 2966 3190 \n",
"Q 3341 2797 3341 1997 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
"L 1178 4863 \n",
"L 1178 0 \n",
"L 603 0 \n",
"L 603 4863 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-45\"/>\n",
" <use xlink:href=\"#DejaVuSans-78\" x=\"63.183594\"/>\n",
" <use xlink:href=\"#DejaVuSans-70\" x=\"122.363281\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"185.839844\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"247.021484\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"310.400391\"/>\n",
" <use xlink:href=\"#DejaVuSans-6e\" x=\"371.923828\"/>\n",
" <use xlink:href=\"#DejaVuSans-74\" x=\"435.302734\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"474.511719\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"502.294922\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"563.574219\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_22\">\n",
" <path d=\"M 49.603125 34.976562 \n",
"L 59.603125 34.976562 \n",
"L 69.603125 34.976562 \n",
"\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"text_12\">\n",
" <!-- Degree 1 Taylor Series -->\n",
" <g transform=\"translate(77.603125 38.476562) scale(0.1 -0.1)\">\n",
" <defs>\n",
" <path id=\"DejaVuSans-44\" d=\"M 1259 4147 \n",
"L 1259 519 \n",
"L 2022 519 \n",
"Q 2988 519 3436 956 \n",
"Q 3884 1394 3884 2338 \n",
"Q 3884 3275 3436 3711 \n",
"Q 2988 4147 2022 4147 \n",
"L 1259 4147 \n",
"z\n",
"M 628 4666 \n",
"L 1925 4666 \n",
"Q 3281 4666 3915 4102 \n",
"Q 4550 3538 4550 2338 \n",
"Q 4550 1131 3912 565 \n",
"Q 3275 0 1925 0 \n",
"L 628 0 \n",
"L 628 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-67\" d=\"M 2906 1791 \n",
"Q 2906 2416 2648 2759 \n",
"Q 2391 3103 1925 3103 \n",
"Q 1463 3103 1205 2759 \n",
"Q 947 2416 947 1791 \n",
"Q 947 1169 1205 825 \n",
"Q 1463 481 1925 481 \n",
"Q 2391 481 2648 825 \n",
"Q 2906 1169 2906 1791 \n",
"z\n",
"M 3481 434 \n",
"Q 3481 -459 3084 -895 \n",
"Q 2688 -1331 1869 -1331 \n",
"Q 1566 -1331 1297 -1286 \n",
"Q 1028 -1241 775 -1147 \n",
"L 775 -588 \n",
"Q 1028 -725 1275 -790 \n",
"Q 1522 -856 1778 -856 \n",
"Q 2344 -856 2625 -561 \n",
"Q 2906 -266 2906 331 \n",
"L 2906 616 \n",
"Q 2728 306 2450 153 \n",
"Q 2172 0 1784 0 \n",
"Q 1141 0 747 490 \n",
"Q 353 981 353 1791 \n",
"Q 353 2603 747 3093 \n",
"Q 1141 3584 1784 3584 \n",
"Q 2172 3584 2450 3431 \n",
"Q 2728 3278 2906 2969 \n",
"L 2906 3500 \n",
"L 3481 3500 \n",
"L 3481 434 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
"Q 2534 3019 2420 3045 \n",
"Q 2306 3072 2169 3072 \n",
"Q 1681 3072 1420 2755 \n",
"Q 1159 2438 1159 1844 \n",
"L 1159 0 \n",
"L 581 0 \n",
"L 581 3500 \n",
"L 1159 3500 \n",
"L 1159 2956 \n",
"Q 1341 3275 1631 3429 \n",
"Q 1922 3584 2338 3584 \n",
"Q 2397 3584 2469 3576 \n",
"Q 2541 3569 2628 3553 \n",
"L 2631 2963 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-54\" d=\"M -19 4666 \n",
"L 3928 4666 \n",
"L 3928 4134 \n",
"L 2272 4134 \n",
"L 2272 0 \n",
"L 1638 0 \n",
"L 1638 4134 \n",
"L -19 4134 \n",
"L -19 4666 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-79\" d=\"M 2059 -325 \n",
"Q 1816 -950 1584 -1140 \n",
"Q 1353 -1331 966 -1331 \n",
"L 506 -1331 \n",
"L 506 -850 \n",
"L 844 -850 \n",
"Q 1081 -850 1212 -737 \n",
"Q 1344 -625 1503 -206 \n",
"L 1606 56 \n",
"L 191 3500 \n",
"L 800 3500 \n",
"L 1894 763 \n",
"L 2988 3500 \n",
"L 3597 3500 \n",
"L 2059 -325 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-53\" d=\"M 3425 4513 \n",
"L 3425 3897 \n",
"Q 3066 4069 2747 4153 \n",
"Q 2428 4238 2131 4238 \n",
"Q 1616 4238 1336 4038 \n",
"Q 1056 3838 1056 3469 \n",
"Q 1056 3159 1242 3001 \n",
"Q 1428 2844 1947 2747 \n",
"L 2328 2669 \n",
"Q 3034 2534 3370 2195 \n",
"Q 3706 1856 3706 1288 \n",
"Q 3706 609 3251 259 \n",
"Q 2797 -91 1919 -91 \n",
"Q 1588 -91 1214 -16 \n",
"Q 841 59 441 206 \n",
"L 441 856 \n",
"Q 825 641 1194 531 \n",
"Q 1563 422 1919 422 \n",
"Q 2459 422 2753 634 \n",
"Q 3047 847 3047 1241 \n",
"Q 3047 1584 2836 1778 \n",
"Q 2625 1972 2144 2069 \n",
"L 1759 2144 \n",
"Q 1053 2284 737 2584 \n",
"Q 422 2884 422 3419 \n",
"Q 422 4038 858 4394 \n",
"Q 1294 4750 2059 4750 \n",
"Q 2388 4750 2728 4690 \n",
"Q 3069 4631 3425 4513 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
"L 2834 2853 \n",
"Q 2591 2978 2328 3040 \n",
"Q 2066 3103 1784 3103 \n",
"Q 1356 3103 1142 2972 \n",
"Q 928 2841 928 2578 \n",
"Q 928 2378 1081 2264 \n",
"Q 1234 2150 1697 2047 \n",
"L 1894 2003 \n",
"Q 2506 1872 2764 1633 \n",
"Q 3022 1394 3022 966 \n",
"Q 3022 478 2636 193 \n",
"Q 2250 -91 1575 -91 \n",
"Q 1294 -91 989 -36 \n",
"Q 684 19 347 128 \n",
"L 347 722 \n",
"Q 666 556 975 473 \n",
"Q 1284 391 1588 391 \n",
"Q 1994 391 2212 530 \n",
"Q 2431 669 2431 922 \n",
"Q 2431 1156 2273 1281 \n",
"Q 2116 1406 1581 1522 \n",
"L 1381 1569 \n",
"Q 847 1681 609 1914 \n",
"Q 372 2147 372 2553 \n",
"Q 372 3047 722 3315 \n",
"Q 1072 3584 1716 3584 \n",
"Q 2034 3584 2315 3537 \n",
"Q 2597 3491 2834 3397 \n",
"z\n",
"\" transform=\"scale(0.015625)\"/>\n",
" </defs>\n",
" <use xlink:href=\"#DejaVuSans-44\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"77.001953\"/>\n",
" <use xlink:href=\"#DejaVuSans-67\" x=\"138.525391\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"202.001953\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"240.865234\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"302.388672\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"363.912109\"/>\n",
" <use xlink:href=\"#DejaVuSans-31\" x=\"395.699219\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"459.322266\"/>\n",
" <use xlink:href=\"#DejaVuSans-54\" x=\"491.109375\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"535.693359\"/>\n",
" <use xlink:href=\"#DejaVuSans-79\" x=\"596.972656\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"656.152344\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"683.935547\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"745.117188\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"786.230469\"/>\n",
" <use xlink:href=\"#DejaVuSans-53\" x=\"818.017578\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"881.494141\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"943.017578\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"984.130859\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"1011.914062\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"1073.4375\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_23\">\n",
" <path d=\"M 49.603125 49.654687 \n",
"L 59.603125 49.654687 \n",
"L 69.603125 49.654687 \n",
"\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"text_13\">\n",
" <!-- Degree 2 Taylor Series -->\n",
" <g transform=\"translate(77.603125 53.154687) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-44\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"77.001953\"/>\n",
" <use xlink:href=\"#DejaVuSans-67\" x=\"138.525391\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"202.001953\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"240.865234\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"302.388672\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"363.912109\"/>\n",
" <use xlink:href=\"#DejaVuSans-32\" x=\"395.699219\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"459.322266\"/>\n",
" <use xlink:href=\"#DejaVuSans-54\" x=\"491.109375\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"535.693359\"/>\n",
" <use xlink:href=\"#DejaVuSans-79\" x=\"596.972656\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"656.152344\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"683.935547\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"745.117188\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"786.230469\"/>\n",
" <use xlink:href=\"#DejaVuSans-53\" x=\"818.017578\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"881.494141\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"943.017578\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"984.130859\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"1011.914062\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"1073.4375\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_24\">\n",
" <path d=\"M 49.603125 64.332813 \n",
"L 59.603125 64.332813 \n",
"L 69.603125 64.332813 \n",
"\" style=\"fill: none; stroke-dasharray: 1.5,2.475; stroke-dashoffset: 0; stroke: #ff0000; stroke-width: 1.5\"/>\n",
" </g>\n",
" <g id=\"text_14\">\n",
" <!-- Degree 5 Taylor Series -->\n",
" <g transform=\"translate(77.603125 67.832813) scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-44\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"77.001953\"/>\n",
" <use xlink:href=\"#DejaVuSans-67\" x=\"138.525391\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"202.001953\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"240.865234\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"302.388672\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"363.912109\"/>\n",
" <use xlink:href=\"#DejaVuSans-35\" x=\"395.699219\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"459.322266\"/>\n",
" <use xlink:href=\"#DejaVuSans-54\" x=\"491.109375\"/>\n",
" <use xlink:href=\"#DejaVuSans-61\" x=\"535.693359\"/>\n",
" <use xlink:href=\"#DejaVuSans-79\" x=\"596.972656\"/>\n",
" <use xlink:href=\"#DejaVuSans-6c\" x=\"656.152344\"/>\n",
" <use xlink:href=\"#DejaVuSans-6f\" x=\"683.935547\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"745.117188\"/>\n",
" <use xlink:href=\"#DejaVuSans-20\" x=\"786.230469\"/>\n",
" <use xlink:href=\"#DejaVuSans-53\" x=\"818.017578\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"881.494141\"/>\n",
" <use xlink:href=\"#DejaVuSans-72\" x=\"943.017578\"/>\n",
" <use xlink:href=\"#DejaVuSans-69\" x=\"984.130859\"/>\n",
" <use xlink:href=\"#DejaVuSans-65\" x=\"1011.914062\"/>\n",
" <use xlink:href=\"#DejaVuSans-73\" x=\"1073.4375\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"pa4fea8ada3\">\n",
" <rect x=\"40.603125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<Figure size 350x250 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Compute the exponential function\n",
"xs = torch.arange(0, 3, 0.01)\n",
"ys = torch.exp(xs)\n",
"\n",
"# Compute a few Taylor series approximations\n",
"P1 = 1 + xs\n",
"P2 = 1 + xs + xs**2 / 2\n",
"P5 = 1 + xs + xs**2 / 2 + xs**3 / 6 + xs**4 / 24 + xs**5 / 120\n",
"\n",
"d2l.plot(xs, [ys, P1, P2, P5], 'x', 'f(x)', legend=[\n",
" \"Exponential\", \"Degree 1 Taylor Series\", \"Degree 2 Taylor Series\",\n",
" \"Degree 5 Taylor Series\"])"
]
},
{
"cell_type": "markdown",
"id": "dcb3fb53",
"metadata": {
"origin_pos": 26
},
"source": [
"Taylor series have two primary applications:\n",
"\n",
"1. *Theoretical applications*: Often when we try to understand a too complex function, using Taylor series enables us to turn it into a polynomial that we can work with directly.\n",
"\n",
"2. *Numerical applications*: Some functions like $e^{x}$ or $\\cos(x)$ are difficult for machines to compute. They can store tables of values at a fixed precision (and this is often done), but it still leaves open questions like \"What is the 1000-th digit of $\\cos(1)$?\" Taylor series are often helpful to answer such questions.\n",
"\n",
"\n",
"## Summary\n",
"\n",
"* Derivatives can be used to express how functions change when we change the input by a small amount.\n",
"* Elementary derivatives can be combined using derivative rules to create arbitrarily complex derivatives.\n",
"* Derivatives can be iterated to get second or higher order derivatives. Each increase in order provides more fine grained information on the behavior of the function.\n",
"* Using information in the derivatives of a single data example, we can approximate well behaved functions by polynomials obtained from the Taylor series.\n",
"\n",
"\n",
"## Exercises\n",
"\n",
"1. What is the derivative of $x^3-4x+1$?\n",
"2. What is the derivative of $\\log(\\frac{1}{x})$?\n",
"3. True or False: If $f'(x) = 0$ then $f$ has a maximum or minimum at $x$?\n",
"4. Where is the minimum of $f(x) = x\\log(x)$ for $x\\ge0$ (where we assume that $f$ takes the limiting value of $0$ at $f(0)$)?\n"
]
},
{
"cell_type": "markdown",
"id": "521bc19a",
"metadata": {
"origin_pos": 28,
"tab": [
"pytorch"
]
},
"source": [
"[Discussions](https://discuss.d2l.ai/t/1088)\n"
]
}
],
"metadata": {
"language_info": {
"name": "python"
},
"required_libs": []
},
"nbformat": 4,
"nbformat_minor": 5
}
Reference in new issue View Git Blame Copy Permalink