Luiz Schons – Algorithms

Dijkstra's Algorithm

Mon, 29 Jul 2024 00:00:00 -0300

Edsger W. Dijkstra was a Dutch computer scientist who made significant contributions to computer science. He is best known for developing Dijkstra's algorithm, which solves the least-cost path problem in a directed or undirected graph with non-negative edges.

But before talking about Dijkstra's algorithm, let's talk about the Breadth-First Search algorithm.

Breadth-First Search

Working with graphs is a common task in computing. One of the most common problems is finding the shortest path between two vertices in a graph. The Breadth-First Search algorithm shows how to find the shortest path in a graph, for example the shortest route between two cities on a map where cities are vertices and roads are edges.

Imagine we need to go from A to G. The Breadth-First Search algorithm helps us find the shortest path. It starts by visiting the source vertex (A) and then explores all neighboring vertices. After that, it explores the vertices two steps away, and so on.

In our case, the shortest path is A -> B -> E -> G.

Here, it takes only 3 steps to reach G.

Dijkstra's Algorithm

But what if the edges have weights? Breadth-First Search no longer works. Dijkstra's algorithm finds the cheapest path in a directed or undirected graph whose edges have weights. It keeps two lists: one of visited vertices and one of unvisited vertices. It selects the unvisited vertex closest to the source, marks it as visited and updates the weights of its neighbors.

Let's adapt the previous example to include weights on the edges.

Now each edge has a weight, and our challenge is to find the path with the lowest total weight.

Let's apply Dijkstra's algorithm to find the cheapest path from A to G.

We start at A and look at its neighbors: reaching B costs 10 and reaching C costs 5.

Vertex Distance from A Path
A 0 A
B 10 A → B
C 5 A → C
The unvisited vertex closest to A is C (5). We visit C and update its neighbors: D now costs 7 (5 + 2) and F now costs 9 (5 + 4).

Vertex Distance from A Path
A 0 A
B 10 A → B
C 5 A → C
D 7 A → C → D
F 9 A → C → F
The next closest is D (7). Through D, E costs 8 (7 + 1). F would cost 13 (7 + 6), which is worse than 9, so F stays the same.

Vertex Distance from A Path
A 0 A
B 10 A → B
C 5 A → C
D 7 A → C → D
F 9 A → C → F
E 8 A → C → D → E
Next is E (8). Through E, G costs 10 (8 + 2). Visiting F and B afterwards does not improve any distance, so the algorithm stops.

Vertex Distance from A Path
A 0 A
B 10 A → B
C 5 A → C
D 7 A → C → D
F 9 A → C → F
E 8 A → C → D → E
G 10 A → C → D → E → G

Vertex	Distance from A	Path
A	0	A
B	10	A → B
C	5	A → C

Vertex	Distance from A	Path
A	0	A
B	10	A → B
C	5	A → C
D	7	A → C → D
F	9	A → C → F

Vertex	Distance from A	Path
A	0	A
B	10	A → B
C	5	A → C
D	7	A → C → D
F	9	A → C → F
E	8	A → C → D → E

Vertex	Distance from A	Path
A	0	A
B	10	A → B
C	5	A → C
D	7	A → C → D
F	9	A → C → F
E	8	A → C → D → E
G	10	A → C → D → E → G

The cheapest path from A to G is A -> C -> D -> E -> G, with a total cost of 10.

Applying it with PHP

Let's build a hands-on example in PHP to apply Dijkstra's algorithm.

$graph = [
 'A' => ['B' => 10, 'C' => 5],
 'B' => ['E' => 8, 'D' => 3],
 'C' => ['D' => 2, 'F' => 4],
 'D' => ['E' => 1, 'F' => 6],
 'E' => ['G' => 2],
 'F' => ['E' => 1],
 'G' => []
];

function dijkstra(array $graph, string $source, string $target): array
{
 $dist = array_fill_keys(array_keys($graph), INF);
 $prev = array_fill_keys(array_keys($graph), null);
 $dist[$source] = 0;

 // SplPriorityQueue returns the highest priority first,
 // so we use the negative distance to get the closest vertex.
 $queue = new SplPriorityQueue();
 $queue->insert($source, 0);

 while (!$queue->isEmpty()) {
 $u = $queue->extract();

 foreach ($graph[$u] as $v => $cost) {
 $alt = $dist[$u] + $cost;
 if ($alt < $dist[$v]) {
 $dist[$v] = $alt;
 $prev[$v] = $u;
 $queue->insert($v, -$alt);
 }
 }
 }

 $path = [];
 for ($u = $target; $u !== null; $u = $prev[$u]) {
 array_unshift($path, $u);
 }

 return $path;
}

$path = dijkstra($graph, 'A', 'G');
echo implode(' -> ', $path); // A -> C -> D -> E -> G

Here is the algorithm, step by step:

We initialize the distances and the priority queue.
We set the distance of the source vertex to 0.
While the priority queue is not empty, we extract the vertex with the smallest distance.
For each neighboring vertex, we compute the alternative distance and update it if it is smaller.
We build the path by walking back through the previous vertices.
We return the path.

Conclusion

With the growing use of AI, Machine Learning and Big Data, graphs have become more and more important. Dijkstra's algorithm is one of the most important algorithms for finding the shortest path in a graph. I hope you now understand how it works and how it can be applied to a real-world scenario. If you liked this article, leave a comment and share it with your friends. See you next time!

Hash Tables

Wed, 06 Mar 2024 00:00:00 -0300

A hash table is a data structure that maps lookup keys to values.

Its main job is to take a simple key and find the value you want quickly and efficiently.

1. Hash Functions

1.1. Introduction

A hash function takes some input data and produces a number that identifies the position of an element in a hash table.

We have already looked at other data structures on this blog, such as finding an element in a linked list or in a binary tree, and we saw that those operations take O(n) and O(log n), respectively.

A hash table, on the other hand, can do that lookup in O(1), that is, in constant time.

This means that no matter how large the table is, the lookup takes the same time.

1.2. Example

A hash function is often used to create an index into an array, which is why it needs to be fast and efficient.

I wrote a very simple hash function that takes a word and returns a number that represents it.

package main

import (
 "fmt"
)

func main() {
 for {
 run()
 }
}

func run(){
 words := [10000]string{}
 fmt.Println("Type a word to hash")
 var word string
 fmt.Scanln(&word)
 wordHash := fake_hash(word)
 fmt.Println("The word", word, "has the hash", wordHash)
 words[wordHash] = word
 fmt.Println("The word", word, "was added to the array at position", wordHash)
 fmt.Println("Type a hash to look up the matching word")
 var hash int
 fmt.Scanln(&hash)
 fmt.Println("The word for hash", hash, "is", words[hash])
}

func fake_hash(word string) int {
 hash := 0
 for i := 0; i < len(word); i++ {
 hash += int(word[i])
 }
 return hash
}

In this example, the fake_hash function takes the word and adds up the ASCII values of its characters, returning a number that represents the word.

But don't worry: you probably won't need to write a hash function yourself, since programming languages already ship with optimized ones.

Your language may call it something else, like a “map” or a “dictionary”, but the concept is the same.

2. Use cases

2.1. Phone Book

A classic example of a hash table is a phone book.

Imagine you want to find someone's phone number.

If the phone book were a linked list, you would have to walk the whole list until you found the person's name.

If it were a binary tree, you would have to walk down the tree until you found the name.

But since the phone book is a hash table, you can find a person's number in constant time, that is, in O(1).

Let's understand this better.

When you add a name and a phone number to the phone book, the hash function is used to generate an index for that name.

When you want to find someone's number, the hash function generates the index for that name, and the phone number is returned.

Like this:

package main

import (
 "fmt"
)
func main() {
 phoneBook := make(map[string]string)
 phoneBook["John"] = "1234-5678"
 phoneBook["Mary"] = "8765-4321"
 phoneBook["Joseph"] = "4321-5678"
 phoneBook["Anna"] = "5678-4321"
 fmt.Println("John's phone number is", phoneBook["John"])
 fmt.Println("Mary's phone number is", phoneBook["Mary"])
 fmt.Println("Joseph's phone number is", phoneBook["Joseph"])
 fmt.Println("Anna's phone number is", phoneBook["Anna"])
}

In this example, the hash function generates an index for each name, and the phone number is returned in constant time.

2.2. DNS

Another classic example of a hash table is DNS (Domain Name System).

DNS is a system that translates Internet domain names into IP addresses.

Imagine you want to visit a website, like www.google.com.

If DNS were not a hash table, you would have to go through the whole list of domains until you found the site's name. Imagine how long that would take.

But since DNS is a hash table, you can find a site's IP address in constant time, that is, in O(1).

Let's look at an example.

package main

import (
 "fmt"
)
func main() {
 dns := make(map[string]string)
 dns["www.google.com"] = "192.168.5.5"
 dns["www.facebook.com"] = "192.168.40.21"
 dns["www.twitter.com"] = "192.168.11.11"
 fmt.Println("The IP address of www.google.com is", dns["www.google.com"])
 fmt.Println("The IP address of www.facebook.com is", dns["www.facebook.com"])
 fmt.Println("The IP address of www.twitter.com is", dns["www.twitter.com"])
}

See how similar these examples are? The idea is the same: the hash table generates an index for each name, and the IP address is returned in constant time.

3. Collisions

3.1. Introduction

A collision happens when two different lookup keys have the same hash value.

But is that possible? Doesn't the hash function generate a unique value for each key?

Well, a hash function does not generate a unique value for each key; it generates a value that represents the key.

And since there are far more possible keys than hash values, two different keys can end up with the same hash value.

3.2. Handling Collisions

There are several ways to handle collisions, but the two most common are:

Chaining
Open Addressing

3.2.1. Chaining

With chaining, each entry in the hash table is a linked list.

When a collision happens, the key is added to the corresponding linked list.

This means that if two different keys have the same hash value, they are added to the same linked list.

The main advantage of chaining is that it is simple to implement; the main disadvantage is that it can be inefficient in terms of space and performance.

3.2.2. Open Addressing

With open addressing, when a collision happens, the key is placed in another position of the hash table.

This means that if two different keys have the same hash value, the second key is stored in another position of the table.

The main advantage of open addressing is that it is efficient in terms of space and performance; the main disadvantage is that it is more complex to implement.

4. Performance

4.1. Load Factor

The load factor is the ratio between the number of keys and the number of positions in the hash table.

The higher the load factor, the higher the chance of collisions.

That is why it is important to keep the load factor low to ensure good hash table performance.

An ideal load factor is below 0.7, meaning less than 70% of the table's positions are occupied.

4.1.1. How to Calculate the Load Factor

The load factor is calculated like this:

NK = number of keys
NP = number of positions in the hash table

NK / NP = load factor

For example, if the hash table has 100 positions and 70 keys, the load factor is 0.7.

70 / 100 = 0.7

4.2. Resizing

When the load factor goes above 0.7, the hash table needs to be resized.

Resizing works like this:

Create a new hash table twice the size of the original
Add all keys from the original table to the new one
Discard the original table

Resizing a hash table is an expensive operation, but it is necessary to keep its performance good.

5. SHA

SHA (Secure Hash Algorithm) is a family of cryptographic hash functions.

It is a great hash function because it produces a practically unique value for each key.

SHA is widely used in cryptography, information security and authentication.

SHA is a very secure hash function, and it is practically impossible to find two different keys with the same hash value.

One of SHA's main advantages is that it is fast, efficient and one-way: it is easy to compute the hash of a key, but practically impossible to recover the key from the hash.

Here is an example of using SHA in Go:

package main
import (
 "crypto/sha256"
 "fmt"
)

func main() {
 word := "hello"
 hash := sha256.Sum256([]byte(word))
 fmt.Printf("The hash of %s is %x\n", word, hash)
}

In this example, the SHA-256 hash function is used to generate a hash value for the word “hello”.

But if you try to use SHA-256 to get the word “hello” back from the hash value, you won't be able to.

6. Conclusion

Hash tables are very efficient data structures for fast lookups.

They are widely used in real-world applications, such as phone books, DNS, cryptography and information security.

You learned about hash functions, use cases, collisions, performance and SHA.

Never write your own hash function and use it in production; always use the ready-made, optimized hash functions from your programming language.

I hope you learned a lot about hash tables and that you can apply this knowledge in your real-world applications.

If you have any questions or suggestions, leave them in the comments.

See you next time!

Quicksort

Wed, 28 Feb 2024 00:00:00 -0300

Quicksort is a very efficient sorting algorithm, invented by C.A.R. Hoare in 1960. It is widely used to sort arrays. The algorithm presented below is a recursive version of Quicksort that picks an element as the pivot and partitions the array so that all elements smaller than the pivot come before it and the larger ones come after it. The subarrays are then sorted recursively.

But before we start talking about the algorithm, you need to understand what D&C (Divide and Conquer) methods are.

D&C methods

D&C methods are algorithms that solve a problem by splitting it into subproblems, solving the subproblems recursively and combining their solutions to solve the original problem.

Let's look at an example of an algorithm that uses D&C.

Example

Imagine you have an array of integers and want to know the sum of all its elements. You can solve this problem with D&C like this:

Array: [1, 2, 3, 4, 5]

What is the base case? The base case is when the array has only one element. In that case, the sum is the element itself.

How do we reach the base case? We remove one element at a time and see what is left.

Removed element	Remaining array
1	[2, 3, 4, 5]
2	[3, 4, 5]
3	[4, 5]
4	[5]

Now that we have reached the base case, let's solve the problem. The sum of all elements of the array [5] is 5. Now we add the 4 we removed earlier. The sum of all elements of the array [4, 5] is 9. Now we add the 3 we removed earlier. The sum of all elements of the array [3, 9] is 12. And so on.

Base case	Sum
[5]	5
[4, 5]	9
[3, 9]	12
[2, 12]	14
[1, 14]	15

The sum of all elements of the array [1, 2, 3, 4, 5] is 15.

This is a simple case of D&C, and many algorithms use this concept to solve problems.

Why not use a loop?

You may be wondering why not just use a loop to solve this problem. And the answer is: you can! But in some cases, the D&C solution is simpler and easier to understand.

Another important point is that some languages, especially functional ones, have no loop constructs (for, while, etc.). In those, the only way to solve such a problem is with D&C.

Once you understand this approach, you can work with other programming languages and understand how they work.

If you have any questions, look for more examples and try solving problems using D&C.

Quicksort

Now that you understand what D&C is, let's talk about Quicksort.

Example

Let's see an example of how Quicksort works.

Array
3, 6, 8, 10, 1, 2, 1

Step 1: Pick an element as the pivot. Let's pick 6.

Step 2: Partition the array so that all elements smaller than the pivot come before it and the larger ones come after it.

Less than 6	Pivot	Greater than 6
3, 1, 2, 1	6	8, 10

Step 3: Sort the subarrays recursively.

What does sorting the subarrays recursively mean? It means repeating steps 1 and 2 for each subarray, picking a pivot and partitioning it.

Let's pick 1 as the pivot.

Less than 1	Pivot	Greater than 1
	1, 1	2, 3

Let's pick 2 as the pivot.

Less than 2	Pivot	Greater than 2
	2	3

Now we have to pick 3 as the pivot.

Less than 3	Pivot	Greater than 3
	3

Now that we have reached the base case, let's sort the subarrays recursively.

We go back up through the arrays until the left side is sorted.

Less than 6	Pivot	Greater than 6
1, 1, 2, 3	6	8, 10

Now let's sort the right side.

Array
8, 10

Step 1: Pick an element as the pivot. Let's pick 10.

Step 2: Partition the array so that all elements smaller than the pivot come before it and the larger ones come after it.

Less than 10	Pivot	Greater than 10
8	10

Now that we have reached the base case, let's sort the subarrays recursively.

We go back up through the arrays until the right side is sorted.

Array
8, 10

Now that the subarrays are sorted, the original array is sorted.

Array
1, 1, 2, 3, 6, 8, 10

That is an example of how Quicksort works.

Performance

In software development, performance is a very important factor, and we often use Big O notation to describe an algorithm's efficiency.

If you are not familiar with Big O notation, I recommend an article I wrote about it (in Portuguese): Big O Notation.

Quicksort's performance is O(n log n) in the best case and O(n²) in the worst case.

Performance depends on the pivot you choose. If you pick a pivot that is the smallest or largest element of the array, Quicksort runs in O(n²). But if you pick a pivot that is the middle element, Quicksort runs in O(n log n).

So how do you choose the pivot? There are several ways, and the right choice depends on your problem. One option is to pick the middle element of the array.

With that in mind, I created a GitHub repository with a Quicksort implementation in Go. If you want to see it, click here.

This article is an introduction to Quicksort, and I hope you now understand how the algorithm works. Next time you hear someone talking about Quicksort, you will know what it is about.

Remember that practice makes perfect, and the more you practice, the more you will understand the subject.

If you have any questions, look for more examples and try solving problems using Quicksort.

I hope you enjoyed the article. See you next time!