Update slides

slides/applications.qmd

@@ -1,27 +1,27 @@
 ---
 title: "Climate Machine Learning Applications"
-subtitle: "ICCS Summer school 2023"
+subtitle: "NCAS Summer school 2025"
 format:
   revealjs:
-    embed-resources: true
+    embed-resources: false
     slide-number: true
     chalkboard: false
     preview-links: auto
     history: false
     highlight-style: monokai
     code-line-numbers: false
-    logo: https://iccs.cam.ac.uk/sites/iccs.cam.ac.uk/files/logo2_2.png
+    logo: https://iccs.cam.ac.uk/sites/default/files/logo2_2.png
     render-on-save: true
     theme: [dark, custom.scss]
 revealjs-plugins:
   - attribution
 authors:
-  - name: <b><u>Jack Atkinson</u></b>
-    orcid: 0000-0001-5001-4812
+  - name: Isaac Akanho
+    orcid:
+    affiliations: ICCS/Cambridge
+  - name: Joe Wallwork
+    orcid:
     affiliations: ICCS/Cambridge
-  - name: Jim Denholm
-    orcid: 0000-0002-2389-3134
-    affiliations: Cambridge
 date: last-modified
 bibliography: references.bib
 ---
@@ -31,12 +31,7 @@ bibliography: references.bib
 
 ## Teaching Material Recap {.smaller}
 
-Over the ML sessions at the summer school we have learnt about:
-
-- Classification - categorising items based on information
-- Regression - using information to predict another value
-
-using:
+We have learnt the theory behind using:
 
 - ANNs - using input _features_ to make predictions
 - CNNs - using _image-like_ data as an input
@@ -57,7 +52,7 @@ using:
 :::
 
 ::: {.fragment}
-![](https://www.mdpi.com/atmosphere/atmosphere-08-00024/article_deploy/html/images/atmosphere-08-00024-g005.png){.absolute width=70% top=15% left=15%}
+![](https://mdpi.com/atmosphere/atmosphere-08-00024/article_deploy/html/images/atmosphere-08-00024-g005.png){.absolute width=70% top=15% left=15%}
 :::
 
 ::: {.attribution}
@@ -74,20 +69,23 @@ NB: [colah provides an excellent article](https://colah.github.io/posts/2014-10-
 
 ## Applications in geosciences: {.smaller}
 
-See review of @kashinath2021physics
+See review of @kashinath2021physics and @gmd-16-6433-2023
 
 :::: {.columns}
 ::: {.column width="50%"}
 
 - Emulation of existing parameterisations  
   [@espinosa2022machine]  
   <br>
-- Data-driven paramterisations  
+- Data-driven parameterisations  
   [@yuval2020stable; @giglio2018estimating]  
   <br>
 - Downscaling/Upsampling  
   [@harris2022generative]
   <br>
+- Climate Emulators  
+  [@watt2025ace2; @chapman2025camulator; @Dheeshjith_2025]  
+  <br>
 :::
 
 ::: {.column width="50%"}
@@ -98,7 +96,8 @@ See review of @kashinath2021physics
   [@zanna2020data; @ma2021data]  
   <br>  
 - Complete forecasting  
-  [@rasp2020weatherbench; @pathak2022fourcastnet; @bi2022pangu]  
+  [@pathak2022fourcastnet; @bi2022pangu; @nguyen2023climax; @rasp2024weatherbench; @Kochkov_2024; @nathaniel2024chaosbench; @bodnar2025foundation]  
+  <br>
   <br>
 
 :::
@@ -175,7 +174,7 @@ Additional challenges:
     Required for accuracy and stability
 * Language interoperation
 
-![]( https://iccs.cam.ac.uk/sites/iccs.cam.ac.uk/files/logo2_2.png ){.absolute top=30% right=7% style="width: 15%; aspect-ratio: 1 / 1; object-fit: cover; object-position: 0 0;"}
+![]( https://iccs.cam.ac.uk/sites/default/files/logo2_2.png ){.absolute top=30% right=7% style="width: 15%; aspect-ratio: 1 / 1; object-fit: cover; object-position: 0 0;"}
 
 ![]( https://raw.githubusercontent.com/DataWaveProject/DataWaveProject.github.io/master/static/images/logo/logo_square.png ){.absolute width=15% top=50% right=15%}
 
@@ -188,7 +187,7 @@ Additional challenges:
 - Train to predict _'image'_ from coarsened version.
   - Topography?
 
-![](https://www.earthdatascience.org/images/earth-analytics/climate-data/downscale-climate-data-met.jpg)
+![](https://earthdatascience.org/images/earth-analytics/climate-data/downscale-climate-data-met.jpg)
 
 ::: {.attribution}
 Image by [Earth Lab](https://www.earthdatascience.org/courses/use-data-open-source-python/hierarchical-data-formats-hdf/intro-to-MACAv2-cmip5-data/)
@@ -198,10 +197,10 @@ Image by [Earth Lab](https://www.earthdatascience.org/courses/use-data-open-sour
 ## Forecasting {.smaller}
 
 - Time-series
-  - popular use
   - Recurrent Neural Nets
 - Complete weather
-  - FourCastNet, Pangu-Weather, GraphCast
+  - FourCastNet, Pangu-Weather, ClimaX
+  - GraphCast, NeuralGCM, Aurora
 
 ![](https://media.springernature.com/full/springer-static/image/art%3A10.1038%2Fs41586-023-06185-3/MediaObjects/41586_2023_6185_Fig2_HTML.png?as=webp){.absolute bottom=0% left=0% width=48%}
 
@@ -216,6 +215,31 @@ Global image from NVIDIA FourCastNet
 Speaker notes go here.
 :::
 
+## Differentiable Models
+
+- Online training
+  - End-to-end differentiable GCMs
+  - Greater stability
+
+![](https://storage.googleapis.com/gweb-research2023-media/images/NeuralGCM-img1.width-1250.png){.absolute bottom=10% left=10% width=100%}
+
+::: {.attribution}
+Images from Google NeuralGCM
+:::
+
+
+## Foundation Models
+- Pretrained on large amount of heterogeneous climate datasets
+- Aims to learn general purpose representations of dynamic
+- Fine tune for specific forecasting tasks
+- Examples:
+  - ClimaX, Microsoft Aurora 
+- Limitations
+  - Poor accuracy beyond short term forecasts
+  - Predict unrealistic dynamics [@chattopadhyay2023long]
+
+
+
 # Challenges
 
 ## Training data - considerations {.smaller}
@@ -287,10 +311,10 @@ Replacing physics-based components of larger models (emulation or data-driven) r
 - Language interoperation
 - Physical compatibility
 
-![]( https://upload.wikimedia.org/wikipedia/commons/5/55/Mathematical_Bridge_tangents.jpg ){style="border-radius: 50%;" .absolute top=40% left=30% width=40%}
+![]( https://upload.wikimedia.org/wikipedia/commons/thumb/5/55/Mathematical_Bridge_tangents.jpg/250px-Mathematical_Bridge_tangents.jpg ){style="border-radius: 50%;" .absolute top=40% left=30% width=40%}
 
 ::: {.attribution}
-[Mathematical Bridge](https://en.wikipedia.org/wiki/Mathematical_Bridge)
+[Mathematical Bridge](https://upload.wikimedia.org/wikipedia/commons/thumb/5/55/Mathematical_Bridge_tangents.jpg/250px-Mathematical_Bridge_tangents.jpg)
 by [cmglee](https://commons.wikimedia.org/wiki/User:Cmglee)
 used under [CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/deed.en)
 :::
@@ -393,7 +417,7 @@ runtime
 
 ![](https://raw.githubusercontent.com/pytorch/pytorch/main/docs/source/_static/img/pytorch-logo-dark.png){style="background-image: radial-gradient(gray 40%, black);" .absolute bottom=12.5% right=22% height=10%}
 
-![](https://upload.wikimedia.org/wikipedia/commons/thumb/1/11/TensorFlowLogo.svg/696px-TensorFlowLogo.svg.png?20180105010857){.absolute bottom=10% left=82% height=15%}
+![](https://upload.wikimedia.org/wikipedia/commons/1/11/TensorFlowLogo.svg){.absolute bottom=10% left=82% height=15%}
 
 
 
@@ -582,33 +606,29 @@ For more information we can be reached at:
 
 ::: {.column width="50%"}
 
-{{< fa pencil >}} \ Jack Atkinson
+{{< fa pencil >}} \ Isaac Akanho
 
 {{< fa solid person-digging >}} \ [ICCS/UoCambridge](https://iccs.cam.ac.uk/about-us/our-team)
 
-{{< fa solid globe >}} \ [jackatkinson.net](https://jackatkinson.net)
 
-{{< fa solid envelope >}} \ [jwa34[AT]cam.ac.uk](mailto:jwa34@cam.ac.uk)
+{{< fa solid envelope >}} \ [ia464[AT]cam.ac.uk](mailto:ia464@cam.ac.uk)
 
-{{< fa brands github >}} \ [jatkinson1000](https://github.com/jatkinson1000)
 
-{{< fa brands mastodon >}} \ [\@jatkinson1000\@fosstodon.org](https://fosstodon.org/@jatkinson1000)
 
 :::
 
 ::: {.column width="50%"}
 
-{{< fa pencil >}} \ Jim Denholm
+{{< fa pencil >}} \ Joe Wallwork
 
-{{< fa solid person-digging >}} \ UoCambridge
-
-{{< fa solid globe >}} \ [linkedin](https://uk.linkedin.com/in/jim-denholm-13043b189)
+{{< fa solid person-digging >}} \ [ICCS/UoCambridge](https://iccs.cam.ac.uk/about-us/our-team)
 
-{{< fa solid envelope >}} \ [jd949[AT]cam.ac.uk](mailto:jd949@cam.ac.uk)
+{{< fa solid envelope >}} \ [jw2423[AT]cam.ac.uk](mailto:jw2423@cam.ac.uk)
 
-{{< fa brands github >}} \ [jdenholm](https://github.com/jdenholm)
 
 :::
 ::::
 
+<!--
 You can also contact the ICCS, [make a resource allocation request](https://iccs.cam.ac.uk/resources-vesri-members/resource-allocation-process), or visit us at the [Summer School RSE Helpdesk](https://docs.google.com/spreadsheets/d/1WKZxp3nqpXrIRMRkfFzc71sos-UD-Uy1zeab0c1p7Xc/edit#gid=0).
+-->

slides/references.bib

@@ -178,3 +178,102 @@ @article{zanna2020data
   year={2020},
   publisher={Wiley Online Library}
 }
+
+@article{Kochkov_2024,
+   title={Neural general circulation models for weather and climate},
+   volume={632},
+   ISSN={1476-4687},
+   url={http://dx.doi.org/10.1038/s41586-024-07744-y},
+   DOI={10.1038/s41586-024-07744-y},
+   number={8027},
+   journal={Nature},
+   publisher={Springer Science and Business Media LLC},
+   author={Kochkov, Dmitrii and Yuval, Janni and Langmore, Ian and Norgaard, Peter and Smith, Jamie and Mooers, Griffin and Klöwer, Milan and Lottes, James and Rasp, Stephan and Düben, Peter and Hatfield, Sam and Battaglia, Peter and Sanchez-Gonzalez, Alvaro and Willson, Matthew and Brenner, Michael P. and Hoyer, Stephan},
+   year={2024},
+   month=jul, pages={1060–1066} }
+
+@Article{gmd-16-6433-2023,
+AUTHOR = {de Burgh-Day, C. O. and Leeuwenburg, T.},
+TITLE = {Machine learning for numerical weather and climate modelling: a review},
+JOURNAL = {Geoscientific Model Development},
+VOLUME = {16},
+YEAR = {2023},
+NUMBER = {22},
+PAGES = {6433--6477},
+URL = {https://gmd.copernicus.org/articles/16/6433/2023/},
+DOI = {10.5194/gmd-16-6433-2023}
+}
+
+@article{watt2025ace2,
+  title={ACE2: accurately learning subseasonal to decadal atmospheric variability and forced responses},
+  author={Watt-Meyer, Oliver and Henn, Brian and McGibbon, Jeremy and Clark, Spencer K and Kwa, Anna and Perkins, W Andre and Wu, Elynn and Harris, Lucas and Bretherton, Christopher S},
+  journal={npj Climate and Atmospheric Science},
+  volume={8},
+  number={1},
+  pages={205},
+  year={2025},
+  publisher={Nature Publishing Group UK London}
+}
+
+@article{chapman2025camulator,
+  title={CAMulator: Fast emulation of the community atmosphere model},
+  author={Chapman, William E and Schreck, John S and Sha, Yingkai and Gagne II, David John and Kimpara, Dhamma and Zanna, Laure and Mayer, Kirsten J and Berner, Judith},
+  journal={arXiv preprint arXiv:2504.06007},
+  year={2025}
+}
+
+@article{Dheeshjith_2025,
+   title={Samudra: An AI Global Ocean Emulator for Climate},
+   volume={52},
+   ISSN={1944-8007},
+   url={http://dx.doi.org/10.1029/2024GL114318},
+   DOI={10.1029/2024gl114318},
+   number={10},
+   journal={Geophysical Research Letters},
+   publisher={American Geophysical Union (AGU)},
+   author={Dheeshjith, Surya and Subel, Adam and Adcroft, Alistair and Busecke, Julius and Fernandez‐Granda, Carlos and Gupta, Shubham and Zanna, Laure},
+   year={2025},
+   month=may }
+
+@article{rasp2024weatherbench,
+  title={WeatherBench 2: A benchmark for the next generation of data-driven global weather models},
+  author={Rasp, Stephan and Hoyer, Stephan and Merose, Alexander and Langmore, Ian and Battaglia, Peter and Russell, Tyler and Sanchez-Gonzalez, Alvaro and Yang, Vivian and Carver, Rob and Agrawal, Shreya and others},
+  journal={Journal of Advances in Modeling Earth Systems},
+  volume={16},
+  number={6},
+  pages={e2023MS004019},
+  year={2024},
+  publisher={Wiley Online Library}
+}
+
+@article{nathaniel2024chaosbench,
+  title={Chaosbench: A multi-channel, physics-based benchmark for subseasonal-to-seasonal climate prediction},
+  author={Nathaniel, Juan and Qu, Yongquan and Nguyen, Tung and Yu, Sungduk and Busecke, Julius and Grover, Aditya and Gentine, Pierre},
+  journal={Advances in Neural Information Processing Systems},
+  volume={37},
+  pages={43715--43729},
+  year={2024}
+}
+
+@article{nguyen2023climax,
+  title={Climax: A foundation model for weather and climate},
+  author={Nguyen, Tung and Brandstetter, Johannes and Kapoor, Ashish and Gupta, Jayesh K and Grover, Aditya},
+  journal={arXiv preprint arXiv:2301.10343},
+  year={2023}
+}
+
+@article{bodnar2025foundation,
+  title={A foundation model for the Earth system},
+  author={Bodnar, Cristian and Bruinsma, Wessel P and Lucic, Ana and Stanley, Megan and Allen, Anna and Brandstetter, Johannes and Garvan, Patrick and Riechert, Maik and Weyn, Jonathan A and Dong, Haiyu and others},
+  journal={Nature},
+  pages={1--8},
+  year={2025},
+  publisher={Nature Publishing Group UK London}
+}
+
+@article{chattopadhyay2023long,
+  title={Long-term instabilities of deep learning-based digital twins of the climate system: The cause and a solution},
+  author={Chattopadhyay, Ashesh and Hassanzadeh, Pedram},
+  journal={arXiv preprint arXiv:2304.07029},
+  year={2023}
+}