Concurrency is not Parallelism

This blog is inspired by a talk by Rob Pike titled "Concurrency is not Parallelism". The talk is available on YouTube and I highly recommend watching it here.

It is a very common misconception that concurrency and parallelism are the same thing. They are not. Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once. In this blog, we will deep dive into what is concurrency, what is parallelism and how they differ. Lets get started!

The Modern World is Parallel

In today's world, almost every computer has multiple cores. This hardware evolution means that to fully utilize the processor, software must be capable of executing multiple instructions at the same time, leveraging parallelism. Whether we're looking at desktop applications, web servers, or even embedded systems, the importance of harnessing parallelism can't be overstated.

Many Languages Support Concurrency

Many modern programming languages offer built-in support for writing concurrent programs. Examples include:

• Go: Known for its simplicity and efficiency, with goroutines and channels.
• Java: Provides extensive threading capabilities through the java.util.concurrent package.
• Python: Supports concurrency via threading, asyncio, and multiprocessing modules.
• C#: Uses the async and await keywords along with the .NET Task Parallel Library (TPL).

With these languages, developers can write programs that handle multiple tasks at the same time, making them responsive and efficient.

Concurrency

Concurrency is the composition of independently executing processes. Here, we aim to structure a system to handle multiple tasks that might not necessarily execute simultaneously. Concurrency is more about the structure of the program—how well it manages multiple tasks and can switch between them effectively.

In Go, for instance, we use goroutines for concurrency:

Code.go
package main

import (
	"fmt"
	"time"
)

func printNumbers() {
	for i := 1; i <= 5; i++ {
		time.Sleep(1 * time.Second)
		fmt.Printf("%d\n", i)
	}
}

func main() {
	go printNumbers()
	time.Sleep(6 * time.Second)
	fmt.Println("Done")
}

This simple program uses a goroutine to print numbers concurrently with the main execution flow.

Parallelism

Parallelism is about executing multiple tasks simultaneously. It's a subset of concurrency and involves actual simultaneous execution on multiple processors or cores.

For example, using Go's runtime.GOMAXPROCS to control the number of OS threads allocated for the Go program:

Code.go
package main

import (
	"fmt"
	"runtime"
	"sync"
)

func printNumbers(wg *sync.WaitGroup) {
	defer wg.Done()
	for i := 1; i <= 5; i++ {
		fmt.Printf("%d\n", i)
	}
}

func main() {
	runtime.GOMAXPROCS(2)
	var wg sync.WaitGroup

	wg.Add(2)
	go printNumbers(&wg)
	go printNumbers(&wg)
	wg.Wait()
	fmt.Println("Done")
}

This example sets the maximum number of OS threads to 2, allowing true parallel execution if running on a multi-core processor.

Concurrency vs Parallelism

To summarize:

• Concurrency is about dealing with lots of things at once. It's a way to structure your program to manage multiple tasks that can start, run, and complete in overlapping time periods.
• Parallelism is about doing lots of things at once. It involves executing multiple tasks literally at the same time using multiple processors or cores.

The confusion between the two often arises because concurrent systems can be parallel, but they don’t have to be. Similarly, parallel systems are inherently concurrent, but they utilize hardware to achieve simultaneous task execution.

Understanding these concepts can significantly improve how you design and implement programs, ensuring they are both responsive and efficient.

Remember to check out Rob Pike’s talk to get a deeper understanding and context regarding these important concepts!

Thanks for reading! Happy Coding! 🚀