Roosta S. Parallel Processing and Parallel Algorithms 2000

Textbook in PDF format

Motivation It is now possible to build powerful single-processor and multiprocessor systems and use them efficiently for data processing, which has seen an explosive expansion in many areas of computer science and engineering. One approach to meeting the performance requirements of the applications has been to utilize the most powerful single-processor system that is available. When such a system does not provide the performance requirements, pipelined and parallel processing structures can be employed. The concept of parallel processing is a departure from sequential processing. In sequential computation one processor is involved and performs one operation at a time. On the other hand, in parallel computation several processors cooperate to solve a problem, which reduces computing time because several operations can be carried out simultaneously. Using several processors that work together on a given computation illustrates a new paradigm in computer problem solving which is completely different from sequential processing. From the practical point of view, this provides sufficient justification to investigate the concept of parallel processing and related issues, such as parallel algorithms. Parallel processing involves utilizing several factors, such as parallel architectures, parallel algorithms, parallel programming languages and performance analysis, which are strongly interrelated. In general, four steps are involved in performing a computational problem in parallel. The first step is to understand the nature of computations in the specific application domain.
Computer Architecture
Components of Parallel Computers
Principles of Parallel Programming
Parallel Programming Approaches
Principles of Parallel Algorithm Design
Parallel Graph Algorithms
Parallel Search Algorithms
Parallel Computational Algorithms
Data Flow and Functional Programming
Asynchronous Parallel Programming
Data Parallel Programming
Artificial Intelligence and Parallel Processing