What is Parallel Programming?
So what is parallel programming? The idea is that it is the use of multiple computer resources in order to solve a computational problem. The resources used can be:
- A set of networked computers.
- A single computer with multiple processors.
- Processors over a network.
Shared Memory
The main benefit of solving problems in a parallel manner are mainly faster computing times by utilizing all of the available resources for a computation. This means that multiple processors have access to the same memory, so they can change all variables within the same execution of the program.
Advantages
- Global address space simplified programming
- Allow incremental parallelization
- Data sharing between CPUs fast and uniform
Disadvantages
- Lack of memory scalability between memory and CPU
- Increasing CPUs increase memory traffic geometrically on shared memory-CPU paths.
- Programmers responsible for synchronization of memory accesses
How to implement a parallel programming solution
Threading is used to implement a parallel solution. Whereas a task would take a O(n) amount of time. A threaded task would take a O(1) amount of time. Meaning that it is roughly the same for the threaded solution .
Problems associated with threading
There are quite a few problems associated with making programs thread safe. The two most important issues are mutual exclusion, Two Threads should not have access to the same resource at the same time, and race conditions. A race condition occurs when a thread needs another resource to be evaluated before the thread is run. If the thread using the resource tries to use it at the same time, this would most likely result in the program to crash, or to output the wrong number.
Although this wasn’t very informational, it is just a very brief introduction to mechanisms used to make code run in a non sequential matter.
