IIT Delhi · Computer Science

Introduction to Parallel Programming in OpenMP

Prof. Yogish Sabharwal · 84 free video lectures

84 video lectures. Topics: Introduction to Parallel Programming; Parallel Architectures and Programming Models; Pipelining; Superpipelining and VLIW; Memory Latency; Cache and Temporal Locality; Cache, Memory bandwidth and Spatial Locality; Intuition for Shared and Distributed Memory architectures…

▶ Play on Open Lecture HallCourse page ↗

Lectures

  1. Introduction to Parallel Programming
  2. Parallel Architectures and Programming Models
  3. Pipelining
  4. Superpipelining and VLIW
  5. Memory Latency
  6. Cache and Temporal Locality
  7. Cache, Memory bandwidth and Spatial Locality
  8. Intuition for Shared and Distributed Memory architectures
  9. Shared and Distributed Memory architectures
  10. Interconnection networks in Distributed Memory architectures
  11. OpenMP: A parallel Hello World Program
  12. Program with Single thread
  13. Program Memory with Multiple threads and Multi-tasking
  14. Context Switching
  15. OpenMP: Basic thread functions
  16. OpenMP: About OpenMP
  17. Shared Memory Consistency Models and the Sequential Consistency Model
  18. Race Conditions
  19. OpenMP: Scoping variables and some race conditions
  20. OpenMP: thread private variables and more constructs
  21. Computing sum: first attempt at parallelization
  22. Manual distribution of work and critical sections
  23. Distributing for loops and reduction
  24. Vector-Vector operations (Dot product)
  25. Matrix-Vector operations (Matrix-Vector Multiply)
  26. Matrix-Matrix operations (Matrix-Matrix Multiply)
  27. Introduction to tasks
  28. Task queues and task execution
  29. Accessing variables in tasks
  30. Completion of tasks and scoping variables in tasks
  31. Recursive task spawning and pitfalls
  32. Understanding LU Factorization
  33. Parallel LU Factorization
  34. Locks
  35. Advanced Task handling
  36. Matrix Multiplication using tasks
  37. The OpenMP Shared Memory Consistency Model
  38. Applications finite element method
  39. Applications deep learning
  40. Introduction to MPI and basic calls
  41. MPI calls to send and receive data
  42. MPI calls for broadcasting data
  43. MPI non blocking calls
  44. Application distributed histogram updation
  45. MPI collectives and MPI broadcast
  46. MPI gathering and scattering collectives
  47. MPI reduction and alltoall collectives
  48. Discussion on MPI collectives design
  49. Characteriziation of interconnects
  50. Linear arrays 2D mesh and torus
  51. d dimensional torus
  52. Hypercube
  53. Trees and cliques
  54. Hockney model
  55. Broadcast and Reduce with recursive doubling
  56. Scatter and Gather with recursive doubling
  57. Reduce scatter and All gather with recursive doubling
  58. Discussion of message sizes in analysis
  59. Revisiting Reduce scatter on 2D mesh
  60. Reduce scatter and Allreduce on the Hypercube
  61. Alltoall on the Hypercube
  62. Lower bounds
  63. Pipeline based algorithm for Allreduce
  64. An improved algorithm for Alltoall on the Hypercube using E cube routing
  65. Pipeline based algorithm for Broadcast
  66. Introduction to parallel graph algorithms
  67. Breadth First Search BFS using matrix algebra
  68. BFS Shared memory parallelization using OpenMP
  69. Distributed memory settings and data distribution
  70. Distributed BFS algorithm
  71. Performance considerations
  72. Prims Algorithm
  73. OpenMP based shared memory parallelization for MST
  74. MPI based distributed memory parallelization for MST
  75. Sequential Algorithm Adaptation from Prims
  76. Parallelization Strategy for Prims algorithm
  77. Dry run with the parallel strategy
  78. Johnsons algorithm with 1D data distribution
  79. Speedup analysis on a grid graph
  80. Floyds algorithm for all pair shortest paths
  81. Floyds algorithm with 2D data distribution
  82. Adaptation to transitive closures
  83. Parallelization strategy for connected components
  84. Analysis for parallel connected components

From NPTEL (IITs and IISc) · CC BY-NC-SA. More from IIT Delhi.

Open Lecture Hall is free, with no ads and no sign-up. It is not run by or connected to any university; the lectures are the universities' own free recordings, played from where they host them. About & rights