Last minute additions

Author: ansantam

fig/auto_diff_comparison_portrait.png

@@ -213,14 +213,15 @@
    "source": [
     "<h2 style=\"color: #b51f2a\">What is automatic differentiation (autodiff)</h2>\n",
     "\n",
+    "In symbolic or numerical differentiation the derivatives are approximated using <u>finite differences</u> and can be very <u>computationally expensive</u> and scale poorly with the number of inputs.\n",
+    "\n",
     "`autodiff` is a way of **calculating gradients** that:\n",
     "\n",
     "- provides exact derivative values\n",
     "- is more efficient than numerical methods ($O(n)$ linear in the cost of computing the value)\n",
     "- can handle complex functions and models\n",
     "- might struggle with non-differentiable functions\n",
-    "\n",
-    "In symbolic or numerical differentiation the derivatives are approximated using <u>finite differences</u> and can be very <u>computationally expensive</u> and scale poorly with the number of inputs.\n"
+    "\n"
    ]
   },
   {
@@ -277,6 +278,8 @@
     "\n",
     "$z = x^2 + 3xy +1 \\ ; \\ u_1 = x^2 \\ ; \\ u_2 = 3xy \\ ; \\ z = u_1 + u_2 + 1$\n",
     "\n",
+    "  <img src=\"fig/auto_diff_comparison_portrait.png\" alt=\"test\" style=\"width:60%; margin:auto; display:block;\"/>\n",
+    "\n",
     "### Forward mode\n",
     "\n",
     "- Propagates derivatives from inputs to outputs in a single forward pass in the computational graph by computing function values and their derivatives simultaneously.\n",
@@ -292,7 +295,9 @@
     "  - $\\frac{\\partial u_2}{\\partial x} = 3y, \\quad \\frac{\\partial u_2}{\\partial y} = 3x $\n",
     "  - Apply the chain rule:\n",
     "  - $\\frac{\\partial z}{\\partial x} = \\frac{\\partial z}{\\partial u_1} \\cdot \\frac{\\partial u_1}{\\partial x} + \\frac{\\partial z}{\\partial u_2} \\cdot \\frac{\\partial u_2}{\\partial x} = 2x + 3y $\n",
-    "  - $\\frac{\\partial z}{\\partial y} = \\frac{\\partial z}{\\partial u_2} \\cdot \\frac{\\partial u_2}{\\partial y} = 3x $\n"
+    "  - $\\frac{\\partial z}{\\partial y} = \\frac{\\partial z}{\\partial u_2} \\cdot \\frac{\\partial u_2}{\\partial y} = 3x $\n",
+    "\n",
+    "  more memory use at the price of speed\n"
    ]
   },
   {
@@ -310,9 +315,10 @@
     "| Forward Mode | Few inputs, many outputs | O(n) per input  | Physics simulations, sensitivity analysis |\n",
     "| Reverse Mode | Many inputs, few outputs | O(n) per output | Deep learning, optimization problems      |\n",
     "\n",
-    "<a href=https://e-dorigatti.github.io/math/deep%20learning/2020/04/07/autodiff.html>\n",
-    "  <img src=\"fig/compgraph.png\" alt=\"test\" style=\"width:60%; margin:auto; display:block;\"/>\n",
-    "</a>\n"
+    "\n",
+    "\n",
+    "  <img src=\"fig/auto_diff_comparison_portrait.png\" alt=\"test\" style=\"width:60%; margin:auto; display:block;\"/>\n",
+    "\n"
    ]
   },
   {
@@ -325,10 +331,11 @@
    "source": [
     "<h2 style=\"color: #b51f2a\">What is autograd?</h2>\n",
     "\n",
-    "`autograd` is the name of a particular autodiff implementation commonly used in:\n",
+    "`autograd` is the name of a particular autodiff implementation and is used interchangeably with `autodiff`.\n",
+    "It's commonly used in:\n",
     "\n",
     "- **Machine learning and deep learning**: frameworks like PyTorch and JAX leverage autograd for gradient-based optimization.\n",
-    "- **Scientific computing**: libraries such as TensorFlow use autodiff for numerical modeling and solving differential equations.\n",
+    "- **Scientific computing**: libraries use autodiff for numerical modeling and solving differential equations.\n",
     "- **Optimization problems**: used in engineering and economics for parameter tuning.\n",
     "- **Physics simulations**: computes gradients in complex simulations like fluid dynamics.\n",
     "- **Probabilistic programming**: helps with Bayesian inference using gradient-based samplers.\n"
@@ -2840,7 +2847,7 @@
  "metadata": {
   "celltoolbar": "Edit Metadata",
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "malapa-cheetah-tutorial-2025",
    "language": "python",
    "name": "python3"
   },