2024-08-29 08:38:50 +02:00
|
|
|
# Parallel Python
|
|
|
|
|
|
2024-08-29 21:15:26 +02:00
|
|
|
A lecture from ASPP 2024. The slides and exercises are in `slides/` and `exercises/`, respectively.
|
2024-08-29 08:38:50 +02:00
|
|
|
|
2024-08-29 21:11:51 +02:00
|
|
|
A really cool example of using `asyncio` with the dakos program is in `extras/`.
|
2024-08-29 08:38:50 +02:00
|
|
|
|
2024-08-29 21:15:26 +02:00
|
|
|
## Lecture outline
|
|
|
|
|
|
2024-08-29 21:18:02 +02:00
|
|
|
1. Processes, threads and THE GIL
|
|
|
|
|
2. Hands-on investigations of embarrassingly parallel problems
|
|
|
|
|
A. Multithreading with NumPy
|
|
|
|
|
B. The multiprocessing package
|
|
|
|
|
C. Blending processes and threads
|
|
|
|
|
3. Going further
|
2024-08-29 21:15:26 +02:00
|
|
|
|
2024-08-29 21:11:51 +02:00
|
|
|
|
|
|
|
|
## Online resources
|
2024-08-29 08:51:07 +02:00
|
|
|
|
|
|
|
|
* Online textbook on operating systems, processes, threads, etc.
|
|
|
|
|
* [Operating Systems: Three Easy Pieces](https://pages.cs.wisc.edu/~remzi/OSTEP/#book-chapters)
|
|
|
|
|
* Chapters 4, 13, and 26 are particularly relevant.
|
|
|
|
|
* But the whole book is interesting.
|
2024-08-29 21:11:51 +02:00
|
|
|
* Report of a talk on Python concurrency types, with focus on sub-interpreters.
|
|
|
|
|
* [Python subinterpreters and free-threading](https://lwn.net/Articles/985041/)
|
|
|
|
|
* Comparison of multithreading, multiprocessing, and asyncio.
|
|
|
|
|
* [Async Python: The Different Forms of Concurrency](http://masnun.rocks/2016/10/06/async-python-the-different-forms-of-concurrency/)
|
|
|
|
|
* Note the code snippets are a little outdated.
|
|
|
|
|
* Example of webscraping with asyncio
|
|
|
|
|
* [Asynchronous Web Scraping in Python - ZenRows](https://www.zenrows.com/blog/asynchronous-web-scraping-python#scrape-multiple-pages-asynchronously)
|
|
|
|
|
* Sub-interpreters in 3.12
|
|
|
|
|
* [Sub-interpreter web workers](https://tonybaloney.github.io/posts/sub-interpreter-web-workers.html )
|
