Mathematics on kenji.blog

Fundamentals and Implementation of Cryptography Using Fermat's Little Theorem

Fri, 11 Sep 2026 22:00:00 +0900

1. Introduction: The Mystery of Mathematics Supporting Modern Cryptography

In modern digital society, especially in communication over the Internet, “encryption” has become an indispensable foundational technology. The reason we can securely browse websites via HTTPS, perform financial transactions through online banking, and exchange private messages on messaging apps is because cryptographic protocols backed by highly advanced mathematical theories are working behind the scenes. Among them, “Public-Key Cryptography” plays a particularly important role, and its prime representative is RSA cryptography.

The security and correctness of many cryptographic algorithms, including RSA cryptography, depend heavily on a very beautiful and powerful theorem discovered by the 17th-century French mathematician Pierre de Fermat. That is Fermat’s Little Theorem. Furthermore, Leonhard Euler’s theorem, which generalizes this, also plays a decisive role in cryptographic theory.

In this article, we will thoroughly explain from the basics how the pure mathematical discovery of Fermat’s Little Theorem is applied to modern practical cryptographic technologies, especially “primality testing” and “RSA cryptography”. This will be a highly detailed technical guide covering mathematical proofs, encryption and decryption mechanisms, and specific algorithm implementations using C++ and Python.

2. Fundamentals of Congruences and Modular Arithmetic

To understand Fermat’s Little Theorem, you first need to become familiar with the mathematical concept of “modular arithmetic (congruence)”. Modular arithmetic is a calculation system that focuses on the “remainder” when dividing by a certain fixed number (called the modulus). Because it’s a calculation like a clock face (which cycles every 12 hours), it is also called “clock mathematics”.

When the remainders of dividing integers $a$ and $b$ by a positive integer $n$ are equal, it is described mathematically as follows:

$$ a \equiv b \pmod n $$

This is read as “$a$ and $b$ are congruent modulo $n$”. For example, the remainder of 17 divided by 5 is 2, and the remainder of 12 divided by 5 is also 2. Therefore, it can be written as:

$$ 17 \equiv 12 \pmod 5 \equiv 2 \pmod 5 $$

In modular arithmetic, normal basic arithmetic operations (addition, subtraction, multiplication) apply as they are.

Addition: If $a \equiv b \pmod n$ and $c \equiv d \pmod n$, then $a + c \equiv b + d \pmod n$
Subtraction: If $a \equiv b \pmod n$ and $c \equiv d \pmod n$, then $a - c \equiv b - d \pmod n$
Multiplication: If $a \equiv b \pmod n$ and $c \equiv d \pmod n$, then $a \times c \equiv b \times d \pmod n$
Exponentiation: If $a \equiv b \pmod n$, then for any natural number $k$, $a^k \equiv b^k \pmod n$

However, care must be taken with division. In general, just because $a \times c \equiv b \times c \pmod n$, you cannot divide both sides by $c$ to get $a \equiv b \pmod n$. This only holds true when $c$ and $n$ are coprime (their greatest common divisor is 1). This concept of “modular inverse” becomes extremely important in the key generation of RSA cryptography discussed later.

3. Mathematical Background and Proof of Fermat’s Little Theorem

Having grasped the basics of modular arithmetic, let’s look at the main subject, Fermat’s Little Theorem.

3.1 Definition of the Theorem

Fermat’s Little Theorem is formulated as follows:

Fermat’s Little Theorem Let $p$ be a prime number, and let $a$ be any integer that is not a multiple of $p$ (i.e., $a$ and $p$ are coprime). Then, the following congruence holds:
$$ a^{p-1} \equiv 1 \pmod p $$

It is also common to express it in a form that holds for all integers $a$ by removing the condition that “$a$ is not a multiple of $p$”. In that case, multiplying both sides by $a$ gives:

$$ a^p \equiv a \pmod p $$

3.2 Verification with Concrete Examples

Let’s check if the theorem really holds using specific numbers. Let the prime number $p = 5$. $p-1 = 4$. We choose an integer $a$ that is not a multiple of $p$.

For $a = 2$: $2^{5-1} = 2^4 = 16$. $16 \div 5 = 3$ remainder $1$. Thus $16 \equiv 1 \pmod 5$. (Holds)
For $a = 3$: $3^{5-1} = 3^4 = 81$. $81 \div 5 = 16$ remainder $1$. Thus $81 \equiv 1 \pmod 5$. (Holds)
For $a = 4$: $4^{5-1} = 4^4 = 256$. $256 \div 5 = 51$ remainder $1$. Thus $256 \equiv 1 \pmod 5$. (Holds)

In this way, no matter what $a$ you choose (as long as it’s not a multiple of 5), raising it to the 4th power and dividing by 5 will always yield a remainder of 1. It seems like magic, but this comes from the beautiful properties that prime numbers possess.

3.3 Mathematical Proof of the Theorem

Why does this happen? Here we introduce an elegant proof using the set of residue classes.

Consider the set $S = \{1, 2, 3, \dots, p-1\}$. These are representatives of integers whose remainders when divided by $p$ are from $1$ to $p-1$. Here, consider a new set $T$ obtained by multiplying each element by an integer $a$ that is coprime to $p$:

$$ T = \{1a, 2a, 3a, \dots, (p-1)a\} $$

Consider the remainder of each element of this set $T$ when divided by $p$. Surprisingly, these remainders, although their order might change, perfectly match the set of elements in the original set $S$. Because:

No element of $T$ can be a multiple of $p$ (since neither $a$ nor the original elements are multiples of $p$).
There are no two distinct elements in $T$ that are congruent modulo $p$. If $ia \equiv ja \pmod p$ ($i \neq j$), since $a$ and $p$ are coprime, we can divide by $a$ to get $i \equiv j \pmod p$, which is a contradiction.

Therefore, the product of all elements of $S$ and the product of all elements of $T$ are congruent modulo $p$.

$$ (1a) \times (2a) \times \dots \times ((p-1)a) \equiv 1 \times 2 \times \dots \times (p-1) \pmod p $$

Simplifying the left side, since there are $p-1$ copies of $a$:

$$ a^{p-1} \cdot (p-1)! \equiv (p-1)! \pmod p $$

Since $(p-1)!$ is coprime to $p$, we can divide both sides by $(p-1)!$, which finally leads to the theorem:

$$ a^{p-1} \equiv 1 \pmod p $$

This is the proof of Fermat’s Little Theorem.

4. Euler’s Totient Function and Euler’s Theorem

Fermat’s Little Theorem is a theorem concerning “prime numbers $p$”, but it was Leonhard Euler who generalized this to “any positive integer $n$”. This extension is essential for understanding RSA cryptography.

4.1 Euler’s Totient Function $\phi(n)$

Euler’s totient function (or Euler’s $\phi$ function) $\phi(n)$ is a function that represents “the number of integers from $1$ to $n$ that are coprime to $n$”.

For a prime number $p$, all integers from $1$ to $p-1$ are coprime to $p$, so $\phi(p) = p - 1$.
For two distinct prime numbers $p, q$, their product $n = p \times q$ has a $\phi(n)$ given by a very simple formula: $$ \phi(p \times q) = \phi(p) \times \phi(q) = (p - 1)(q - 1) $$

This property is the fundamental logic in RSA key generation.

4.2 Euler’s Theorem

Euler generalized Fermat’s Little Theorem as follows:

Euler’s Theorem For a positive integer $n$ and an integer $a$ coprime to it, the following holds:
$$ a^{\phi(n)} \equiv 1 \pmod n $$

If $n$ is a prime number $p$, then $\phi(p) = p - 1$, so this becomes Fermat’s Little Theorem itself ($a^{p-1} \equiv 1 \pmod p$). In other words, Fermat’s Little Theorem is merely a special case of Euler’s Theorem.

5. Finding Giant Prime Numbers: Fermat Primality Test

In cryptographic technologies (such as RSA cryptography and Diffie-Hellman key exchange), it is necessary to find “giant prime numbers” spanning hundreds of digits at high speed. However, to test whether a giant number $N$ is prime, checking if it is divisible by every number from $2$ to $\sqrt{N}$ (trial division) would take as long as the lifespan of the universe.

This is where the Fermat Primality Test comes in, a “probabilistic primality test” that takes advantage of Fermat’s Little Theorem.

5.1 What is a Probabilistic Primality Test?

According to Fermat’s Little Theorem, if $p$ is prime, then for any $a$ ($1 < a < p$), $a^{p-1} \equiv 1 \pmod p$ must hold. Taking the contrapositive, we can say that “if $a^{p-1} \not\equiv 1 \pmod p$ for some $a$, then $p$ is absolutely not a prime number (it is a composite number)”.

Therefore, if we want to determine whether $N$ is prime, we randomly choose several $a$’s, calculate $a^{N-1} \pmod N$, and check if it equals $1$. If an answer other than $1$ appears even once, $N$ is definitively a composite number. If it equals $1$ no matter how many times we try, we can determine with high probability that $N$ is “probably prime”.

5.2 Algorithm Explanation and Flowchart

The algorithm for the Fermat primality test is as follows:

flowchart TD Start["Start"] --> Input["Input number to test p and number of tests k"] Input --> LoopStart["Loop for i = 0 to k-1"] LoopStart --> Condition{"i < k ?"} Condition -- "Yes" --> RandomA["Select random integer a in range 1 < a < p-1"] RandomA --> Calc["Calculate modular exponentiation a^(p-1) mod p"] Calc --> CheckPrime{"Is result 1 ?"} CheckPrime -- "No" --> ReturnComposite["p is a composite number (definite)"] CheckPrime -- "Yes" --> Increment["Increment i"] Increment --> Condition Condition -- "No" --> ReturnPrime["p is probably prime (probabilistic)"] ReturnComposite --> End["End"] ReturnPrime --> End

5.3 The Pitfall of Carmichael Numbers (Pseudoprimes)

While the Fermat test is very fast, it has a significant flaw. There are devilish numbers that are composite numbers but still satisfy $a^{N-1} \equiv 1 \pmod N$ for all $a$. These are called Carmichael numbers. The smallest Carmichael number is $561$ ($3 \times 11 \times 17$).

Because Carmichael numbers exist, a pure Fermat test alone cannot provide absolute primality testing. Therefore, in actual cryptographic systems (such as OpenSSL), the Miller-Rabin primality test, an improved version of the Fermat test, is used as the standard. The Miller-Rabin test can detect Carmichael numbers, effectively reducing the probability of misjudgment to zero.

5.4 Fast Modular Exponentiation (Exponentiation by Squaring)

In the primality testing algorithm, we need to calculate $a^{N-1} \pmod N$, but when $N$ is huge, $a^{N-1}$ becomes an astronomically large number that cannot fit into a computer’s memory. This is solved by Exponentiation by Squaring or modular exponentiation. By taking the modulo ($mod N$) at each step of the calculation, the value is always kept smaller than $N$, allowing it to be calculated very quickly (with a computational complexity of $O(\log N)$).

6. Implementation of Primality Testing and Modular Exponentiation

Now, let’s implement the Fermat primality test and exponentiation by squaring in C++ and Python.

6.1 Implementation in C++

In C++, standard integer types are prone to overflow, so handling giant numbers requires a multiple-precision integer library (like GMP), but here we show an implementation within the range of 64-bit integers (unsigned long long) to understand the algorithm.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51


#include <iostream>
#include <random>

using namespace std;

// Fast modular exponentiation (a^b mod m) - Exponentiation by squaring
unsigned long long power_mod(unsigned long long a, unsigned long long b, unsigned long long m) {
 unsigned long long result = 1;
 a = a % m;
 while (b > 0) {
 // If the lowest bit of b is 1, multiply result by a
 if (b % 2 == 1) {
 result = (__int128)result * a % m; // 128-bit extension to prevent overflow
 }
 // Square a
 a = (__int128)a * a % m;
 // Right shift b (halve it)
 b /= 2;
 }
 return result;
}

// Fermat primality test
bool fermat_is_prime(unsigned long long p, int iterations = 5) {
 if (p <= 1) return false;
 if (p <= 3) return true;
 if (p % 2 == 0) return false;

 random_device rd;
 mt19937_64 gen(rd());
 uniform_int_distribution<unsigned long long> dis(2, p - 2);

 for (int i = 0; i < iterations; ++i) {
 unsigned long long a = dis(gen);
 // If a^(p-1) mod p is not 1, it's composite
 if (power_mod(a, p - 1, p) != 1) {
 return false;
 }
 }
 return true; // Probably prime
}

int main() {
 unsigned long long num = 1000000007; // A known prime
 if (fermat_is_prime(num, 10)) {
 cout << num << " is probably prime." << endl;
 } else {
 cout << num << " is composite." << endl;
 }
 return 0;
}

6.2 Implementation in Python

Python’s standard integer type supports arbitrary precision integers, so there’s no need to worry about overflow. Furthermore, Python’s built-in function pow(a, b, m) internally uses exponentiation by squaring, so it’s very fast.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


import random

def fermat_is_prime(p, iterations=5):
 """
 Probabilistic primality test using Fermat's primality test
 """
 if p <= 1:
 return False
 if p <= 3:
 return True
 if p % 2 == 0:
 return False

 for _ in range(iterations):
 # Choose a random number a between 2 and p-2
 a = random.randint(2, p - 2)
 # Calculate a^(p-1) mod p. Built-in pow is fast.
 if pow(a, p - 1, p) != 1:
 return False # Definitely composite

 return True # Probably prime

# Test
number_to_test = 104729
if fermat_is_prime(number_to_test, 10):
 print(f"{number_to_test} is probably prime.")
else:
 print(f"{number_to_test} is composite.")

7. Application to RSA Cryptography: Where Fermat and Euler Bear Fruit

The greatest application of Fermat’s Little Theorem (and Euler’s Theorem) is RSA cryptography, developed in 1977 by Rivest, Shamir, and Adleman. RSA cryptography is an epoch-making system called “public-key cryptography”, realizing a mechanism where the key for encryption (public key) is published to the whole world, while the key for decryption (private key) is known only to the receiver themselves.

This asymmetry is based on the computational security that “factorizing a giant composite number into its prime factors is extremely difficult.”

7.1 Mechanism of RSA Cryptography (Key Generation, Encryption, Decryption)

Let’s check the overall communication flow of RSA cryptography with a Mermaid sequence diagram.

sequenceDiagram participant Alice["Alice (Receiver)"] participant Bob["Bob (Sender)"] Alice->>Alice: "Generate large primes p, q" Alice->>Alice: "Calculate N = p * q, φ(N) = (p-1)(q-1)" Alice->>Alice: "Calculate public key e and private key d (e*d ≡ 1 mod φ(N))" Alice->>Bob: "Send public key (N, e)" Note over Bob: "Prepare plaintext M (M < N)" Bob->>Bob: "Calculate ciphertext C = M^e mod N" Bob->>Alice: "Send ciphertext C" Alice->>Alice: "Calculate plaintext M = C^d mod N to decrypt"

The mathematical detail steps are explained below.

Step 1: Key Generation (Task of Receiver Alice)

Randomly generate two giant prime numbers $p$ and $q$ (the primality test mentioned above is used here).
Calculate their product $N = p \times q$. This $N$ is made public.
Using Euler’s totient function, calculate $\phi(N) = (p-1)(q-1)$.
Choose an integer $e$ (public exponent) that is coprime to $\phi(N)$ (often $e = 65537$ is used).
Calculate the modular inverse $d$ (private exponent) of $e$. In other words, find $d$ that satisfies: $$ e \cdot d \equiv 1 \pmod{\phi(N)} $$ The extended Euclidean algorithm is used for this calculation.

Now, the public key is $(N, e)$, and the private key is $(N, d)$. ($p, q, \phi(N)$ are immediately discarded or strictly hidden).

Step 2: Encryption (Task of Sender Bob)

Suppose Bob wants to send a message $M$ to Alice ($M$ is a numerically converted character, and $0 \le M < N$). Bob uses Alice’s public key $(N, e)$ to perform the following calculation to create ciphertext $C$.

$$ C \equiv M^e \pmod N $$

He sends this $C$ to Alice over the network.

Step 3: Decryption (Task of Receiver Alice)

Alice, receiving the ciphertext $C$, performs the following calculation using the private key $d$ that only she knows.

$$ M' \equiv C^d \pmod N $$

Surprisingly, this calculation result $M'$ perfectly matches the original message $M$.

7.2 Why can it be decrypted? (Mathematical Proof)

Here, Fermat’s Little Theorem (Euler’s Theorem) shows its true worth. Why does $C^d \pmod N$ return to $M$?

Let’s expand the decryption equation. Since $C \equiv M^e \pmod N$,

$$ C^d \equiv (M^e)^d \equiv M^{ed} \pmod N $$

In the key generation step, we chose $d$ such that $e \cdot d \equiv 1 \pmod{\phi(N)}$. This means there exists an integer $k$ such that it can be written as:

$$ e \cdot d = 1 + k \cdot \phi(N) $$

Substitute this into the above equation:

$$ M^{ed} = M^{1 + k \cdot \phi(N)} = M \cdot M^{k \cdot \phi(N)} = M \cdot (M^{\phi(N)})^k \pmod N $$

Here, Euler’s Theorem ($M^{\phi(N)} \equiv 1 \pmod N$) comes into play. (*Strictly speaking, $M$ and $N$ need to be coprime, but in RSA, the probability that $M$ and $N$ are not coprime is astronomically low, and using the Chinese Remainder Theorem, it can be proven to hold even if they are not coprime).

Applying Euler’s Theorem, since $M^{\phi(N)} \equiv 1$:

$$ M \cdot (1)^k \equiv M \pmod N $$

$M$ is beautifully restored! The properties of numbers discovered hundreds of years ago by Fermat and Euler perfectly guarantee the confidentiality of modern digital communication.

8. Toy Implementation of RSA Cryptography (Python)

It’s hard to get a real feel from theory alone, so let’s actually implement the key generation, encryption, and decryption process of RSA cryptography using Python. This is a “toy implementation” for educational purposes, but the math used is exactly the same as the real thing.

The “extended Euclidean algorithm” for finding the modular inverse $d$ is also included in the implementation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86


import random

# Find the greatest common divisor
def gcd(a, b):
 while b != 0:
 a, b = b, a % b
 return a

# Extended Euclidean algorithm (Find x, y for ax + by = gcd(a,b))
# Used to find d for e*d ≡ 1 (mod φ(N))
def extended_gcd(a, b):
 if a == 0:
 return (b, 0, 1)
 else:
 g, y, x = extended_gcd(b % a, a)
 return (g, x - (b // a) * y, y)

def mod_inverse(e, phi):
 g, x, y = extended_gcd(e, phi)
 if g != 1:
 raise Exception('Inverse does not exist')
 else:
 return x % phi

# Prime number generation function (simplified version: generates small primes)
def generate_prime(bits):
 while True:
 p = random.getrandbits(bits)
 # Simplified check instead of the Fermat test mentioned above
 if p > 1 and pow(2, p-1, p) == 1 and pow(3, p-1, p) == 1:
 return p

# RSA key generation
def generate_keypair(bits=16):
 p = generate_prime(bits)
 q = generate_prime(bits)
 # Ensure p and q are not the same
 while p == q:
 q = generate_prime(bits)

 n = p * q
 phi = (p - 1) * (q - 1)

 # e is often a prime like 65537, but here we choose it randomly
 e = random.randrange(1, phi)
 g = gcd(e, phi)
 while g != 1:
 e = random.randrange(1, phi)
 g = gcd(e, phi)

 # Calculation of private key d
 d = mod_inverse(e, phi)

 # Public key (e, n), Private key (d, n)
 return ((e, n), (d, n))

def encrypt(pk, plaintext):
 e, n = pk
 # Calculate plaintext^e mod n
 cipher = [pow(ord(char), e, n) for char in plaintext]
 return cipher

def decrypt(sk, ciphertext):
 d, n = sk
 # Calculate cipher^d mod n and convert back to characters
 plain = [chr(pow(char, d, n)) for char in ciphertext]
 return ''.join(plain)

# Example execution
if __name__ == '__main__':
 print("--- RSA Cryptography Toy Implementation ---")
 public_key, private_key = generate_keypair(bits=12) # Use 12-bit primes

 print(f"Public key (e, n): {public_key}")
 print(f"Private key (d, n): {private_key}")

 message = "Hello Math!"
 print(f"\nOriginal message: {message}")

 # Encryption
 encrypted_msg = encrypt(public_key, message)
 print(f"Ciphertext: {encrypted_msg}")

 # Decryption
 decrypted_msg = decrypt(private_key, encrypted_msg)
 print(f"Decrypted message: {decrypted_msg}")

When you run this code, you can see how an array of characters is converted into an unfamiliar array of numbers (ciphertext), which is then beautifully restored to the original string by the private key.

9. Conclusion: The Intersection of Mathematical Beauty and Practicality

In the 17th century when Pierre de Fermat discovered this “Little Theorem”, no one thought it would be of any use. Fermat himself studied number theory out of pure mathematical curiosity.

However, about 300 years later in the 1970s, at the dawn of computer networks, Fermat’s theorem made a dramatic comeback as an indispensable cryptographic technology for establishing secure communication protocols. Primality testing technology based on Fermat’s Little Theorem and RSA cryptography based on Euler’s theorem literally support modern Internet infrastructure.

The LINE messages we casually send every day, the shopping on Amazon, all dance on this simple and beautiful formula $a^{p-1} \equiv 1 \pmod p$. No matter how abstract mathematics may be, Fermat’s Little Theorem teaches us that the time will definitely come when it will be useful to humanity.

When studying programming or cryptographic theory, understanding the mathematical structures at their foundation will become a great weapon for deeply understanding the behavior of libraries provided as black boxes and designing more secure systems.

Mathematical Intuition of Lattice-based Cryptography

Fri, 11 Sep 2026 21:00:00 +0900

1. Introduction: The Dawn of Post-Quantum Cryptography (PQC) and the Rise of Lattice-based Cryptography

The digital infrastructure of modern society is supported by public-key cryptography technologies such as RSA cryptography and Elliptic Curve Cryptography (ECC). These cryptographic schemes base their security on the mathematical difficulty of problems like the “prime factorization problem” and the “discrete logarithm problem,” which are believed to be inefficient (requiring exponential time) to solve with conventional classical computers.

However, “Shor’s algorithm,” published by Peter Shor in 1994, sent shockwaves through the cryptographic world. This algorithm mathematically proved that once a large-scale quantum computer is realized, it would be able to solve the prime factorization problem and the discrete logarithm problem in polynomial time. This means that the widely used public-key cryptography of today will become completely decipherable in the future.

To counter such a “Quantum Threat,” research into new cryptographic schemes that are difficult to break even with quantum computers became an urgent task. This field is called “Post-Quantum Cryptography (PQC)” or “quantum-resistant cryptography.”

There are several strong candidates for PQC. Examples include hash-based cryptography, code-based cryptography, multivariate polynomial cryptography, and isogeny-based cryptography. Among them, “Lattice-based cryptography” is currently attracting the most attention and is at the center of the PQC standardization process by NIST (National Institute of Standards and Technology). Compared to other methods, lattice-based cryptography has extremely fast encryption and decryption processing speeds, and it has the outstanding feature of an extremely strong security proof in cryptographic theory: a reduction from “worst-case complexity” to “average-case complexity.”

In this article, starting from the mathematical definition of a “Lattice,” which is the foundation of lattice-based cryptography, we will thoroughly and deeply explain difficult problems on lattices such as SVP (Shortest Vector Problem) and CVP (Closest Vector Problem), and the “LWE (Learning With Errors) problem,” which can be said to be the heart of modern lattice-based cryptography, using mathematical formulas, geometric intuition, and specific numerical examples.

2. Mathematical Definition and Geometric Intuition of a Lattice

2.1 Vector Spaces and Lattices

In mathematics, a “Lattice” is a set of discrete points arranged regularly in an $n$-dimensional real vector space $\mathbb{R}^n$. It is similar to a Vector Space learned in linear algebra, but there is a crucial difference. While a vector space is a continuous space represented by a linear combination of basis vectors with “real coefficients,” a lattice is a discrete space represented by a linear combination of basis vectors with “integer coefficients.”

Let’s give a strict mathematical definition. Consider $n$ ($n \le m$) linearly independent vectors $\mathbf{b}_1, \mathbf{b}_2, \dots, \mathbf{b}_n$ in an $m$-dimensional real vector space $\mathbb{R}^m$. Let a matrix having these vectors as column vectors be $B = [\mathbf{b}_1, \mathbf{b}_2, \dots, \mathbf{b}_n] \in \mathbb{R}^{m \times n}$. This $B$ is called the “Basis” of the lattice.

The lattice $\mathcal{L}(B)$ generated by this basis $B$ is defined as follows:

$$ \mathcal{L}(B) = \left\{ \sum_{i=1}^{n} x_i \mathbf{b}_i \mathrel{\bigg|} x_i \in \mathbb{Z} \right\} = \{ B \mathbf{x} \mid \mathbf{x} \in \mathbb{Z}^n \} $$

What is important here is that the coefficients $x_i$ are limited to integers $\mathbb{Z}$, not real numbers $\mathbb{R}$. As a result, rather than a continuous space with infinitely many points, a “set of discrete points” like equally spaced intersections is formed.

2.2 Geometric Image

Let’s consider an example of a 2-dimensional plane $\mathbb{R}^2$. When $\mathbf{b}_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ and $\mathbf{b}_2 = \begin{pmatrix} 0 \\ 1 \end{pmatrix}$ are chosen as basis vectors, the lattice generated by them is the set of all integer coordinates $(x, y) \in \mathbb{Z}^2$ on the coordinate plane. This is the simplest “square lattice.”

However, lattices are not always orthogonal. For example, considering the basis $\mathbf{b}_1 = \begin{pmatrix} 2 \\ 1 \end{pmatrix}$ and $\mathbf{b}_2 = \begin{pmatrix} 1 \\ 3 \end{pmatrix}$, the generated points become like the intersections of an obliquely skewed mesh.

2.3 Non-uniqueness of the Basis and Unimodular Transformations

There is an important property related to the foundation of the security of lattice-based cryptography. That is, “there are infinitely many bases that generate the same lattice.”

For example, the $\mathbb{Z}^2$ lattice generated by the previous basis $\mathbf{b}_1 = (1, 0)^T, \mathbf{b}_2 = (0, 1)^T$ can be generated as exactly the same lattice $\mathbb{Z}^2$ using the basis $\mathbf{b}'_1 = (1, 1)^T, \mathbf{b}'_2 = (2, 3)^T$.

The necessary and sufficient condition for a basis $B$ and another basis $B'$ to generate the same lattice is that there exists a matrix with integer components $U \in \mathbb{Z}^{n \times n}$ whose determinant is $\det(U) = \pm 1$, and it can be expressed as:

$$ B' = B U $$

Such a matrix $U$ is called a “Unimodular matrix.”

The basic idea in its application to cryptography is to use a “good basis” (a basis that is close to orthogonal and consists of short vectors) as a secret key, and a “bad basis” (a basis that is extremely skewed relative to each other and consists of very long vectors) as a public key. It becomes very difficult to calculate a good basis from a bad basis as the dimension increases. This is the basic intuition behind lattice-based cryptography.

3. Computationally Hard Problems in Lattices

The security of lattice-based cryptography depends on the difficulty of solving specific mathematical problems on lattices. Here, we introduce the two most fundamental and famous problems.

3.1 Shortest Vector Problem (SVP)

SVP is the most classical and famous problem in lattice theory.

Definition (SVP): Given an arbitrary lattice basis $B$, find the vector $\mathbf{v}$ with the minimum Euclidean norm (length) among the non-zero vectors belonging to that lattice $\mathcal{L}(B)$.

Expressed mathematically, it is the problem of finding $\mathbf{v}$ such that $\min_{\mathbf{v} \in \mathcal{L}(B) \setminus \{\mathbf{0}\}} \| \mathbf{v} \|$. This minimum length is written as $\lambda_1(\mathcal{L})$ and is called the “first successive minimum” of the lattice.

In lower dimensions such as 2D or 3D, you can draw a figure and visually find the shortest vector. Alternatively, it can be efficiently solved using algorithms like Gauss’s lattice reduction algorithm. However, when the dimension $n$ becomes a high dimension such as hundreds to thousands, it is known that strictly solving SVP is NP-hard.

In actual cryptography, instead of the strict shortest vector, an approximate SVP ($\gamma$-SVP) is used, which finds an “approximately short vector.” When the approximation factor $\gamma$ is of polynomial size, this problem is still considered very difficult.

3.2 Closest Vector Problem (CVP)

CVP is also an extremely important problem in lattice-based cryptography.

Definition (CVP): Given an arbitrary lattice basis $B$ and an arbitrary target vector $\mathbf{t} \in \mathbb{R}^m$ in space (which is not necessarily a lattice point), find the lattice point $\mathbf{v} \in \mathcal{L}(B)$ that is closest to $\mathbf{t}$ among the lattice points.

Expressed mathematically, it is the problem of searching for a lattice point $\mathbf{v}$ such that $\min_{\mathbf{v} \in \mathcal{L}(B)} \| \mathbf{v} - \mathbf{t} \|$.

Like SVP, CVP is also NP-hard in high dimensions. From the perspective of application to cryptography, the LWE problem described later is closely related to a special variant of this CVP (Bounded Distance Decoding: BDD).

3.3 Why Are They Unsolvable in High Dimensions? (Limits of LLL and BKZ)

A famous algorithm for solving high-dimensional lattice problems is the LLL algorithm (Lenstra-Lenstra-Lovász algorithm). The LLL algorithm operates in polynomial time and can reduce a lattice basis to a “good basis” to some extent. However, since the shortest vector found by the LLL algorithm has an exponential approximation factor ($2^{\mathcal{O}(n)}$) relative to the length of the true shortest vector, it is not enough to break the security of the cryptography.

By using a more powerful basis reduction algorithm such as the BKZ (Block Korkine-Zolotarev) algorithm, which is an improvement over LLL, a shorter vector can be found, but its computational complexity increases exponentially with respect to the block size. In lattice-based cryptography, secure parameters (such as the size of the dimension $n$) are determined by estimating the execution time of this BKZ algorithm. In current PQC standard parameters, values of dimension $n$ from 500 to over 1000 are chosen, and it is said that it would take more than the age of the universe to decrypt even if supercomputers or future quantum computers were used.

4. Mathematical Formulation of the LWE (Learning With Errors) Problem

Most of modern lattice-based cryptography is based on the “LWE (Learning With Errors) problem” proposed by Oded Regev in 2005. The beauty of the LWE problem lies in the simplicity of its formulation and the fact that it has a powerful mathematical proof of “reduction from worst-case to average-case complexity.”

4.1 Systems of Linear Equations Without Noise

To understand the LWE problem, let’s first consider a simple system of linear equations without noise. Suppose there is an unknown secret vector $\mathbf{s} \in \mathbb{Z}_q^n$ (each component is an integer from $0$ to $q-1$). Here, $q$ is assumed to be a prime number.

Choose random coefficient vectors $\mathbf{a}_1, \mathbf{a}_2, \dots \in \mathbb{Z}_q^n$, and calculate their inner product with the secret vector $\mathbf{s}$ modulo $q$. $b_1 = \langle \mathbf{a}_1, \mathbf{s} \rangle \pmod q$ $b_2 = \langle \mathbf{a}_2, \mathbf{s} \rangle \pmod q$ $\vdots$

Given a sufficient number (at least $n$) of pairs $(\mathbf{a}_i, b_i)$, we can easily recover the secret vector $\mathbf{s}$ by using “Gaussian elimination” in linear algebra. This is a problem that can be easily solved in polynomial time.

4.2 Definition of the LWE Problem: Adding Noise

So, what happens if we add a slight “noise (error)” to this problem? This is the essence of the LWE problem.

For an unknown secret vector $\mathbf{s} \in \mathbb{Z}_q^n$, we add a small error $e_i \in \mathbb{Z}_q$ to the result of each equation. $b_i = \langle \mathbf{a}_i, \mathbf{s} \rangle + e_i \pmod q$

Here, $e_i$ is a small integer value with a mean of 0 and a relatively small standard deviation (for example, chosen from a discrete Gaussian distribution, like a normal distribution). The given information is a list of pairs of a random vector $\mathbf{a}_i$ and $b_i$ calculated by adding an error to it. $( \mathbf{a}_1, b_1 ), ( \mathbf{a}_2, b_2 ), \dots, ( \mathbf{a}_m, b_m )$

This becomes very neat when expressed as a matrix. Using a random matrix $A \in \mathbb{Z}_q^{m \times n}$, a secret vector $\mathbf{s} \in \mathbb{Z}_q^n$, and an error vector $\mathbf{e} \in \mathbb{Z}_q^m$, it can be written as:

$$ \mathbf{b} = A \mathbf{s} + \mathbf{e} \pmod q $$

Only $A$ and $\mathbf{b}$ are given. The problem of finding $\mathbf{s}$ from this is the “Search LWE problem.”

Because the error $e_i$ is included, if one tries to use Gaussian elimination, the errors amplify exponentially during the process of adding and subtracting equations, making it impossible to reach the correct answer. At first glance, it looks like a simple system of linear equations, but just by adding this small noise, the difficulty of the problem jumps to an NP-hard level.

4.3 Decision LWE Problem

What is frequently used in cryptographic theory proofs is the “Decision LWE problem,” a variation of the Search LWE problem.

The Decision LWE problem is the problem of determining which of the following two distributions a given list of samples came from:

LWE Distribution: Intentionally calculated $(A, \mathbf{b} = A\mathbf{s} + \mathbf{e} \pmod q)$
Uniform Random Distribution: $(A, \mathbf{u})$ consisting of a completely randomly chosen matrix $A$ and vector $\mathbf{u}$

Surprisingly, if the parameters of the LWE problem are chosen appropriately, the pairs obtained from the LWE distribution become “Computationally Indistinguishable” from pairs of completely random data. This property provides the foundation for LWE-based cryptography to generate “ciphertexts indistinguishable from random numbers.”

4.4 Reduction from Worst-Case to Average-Case Complexity (Regev’s Theorem)

Oded Regev’s greatest achievement is mathematically linking the difficulty of this LWE problem to the difficulty of the aforementioned lattice problems (SVP and CVP).

Using a quantum reduction, he proved that “if there is a polynomial-time algorithm that can solve the LWE problem on average (for randomly chosen $A$ and $\mathbf{e}$), then there is a polynomial-time quantum algorithm that can solve the Gap-SVP for the worst case (the most difficult case) of any lattice.” (Later, a classical reduction was also demonstrated by Peikert et al.)

This is a dream-like property in cryptographic theory. This is because it dispels the concern that “the cipher might be broken because we happened to choose a weak key (a part of the average case),” and gives a strong guarantee that “if average-case LWE can be solved, all hard problems on lattices can be solved (therefore LWE is absolutely hard).”

graph TD A["Worst-case Lattice Problems (Gap-SVP, SIVP)"] -->|Quantum/Classical Reduction| B["Average-case LWE Problem"] B -->|Cryptographic Construction| C["LWE-based Cryptosystems (PKE, KEM, FHE)"] style A fill:#ffcccc,stroke:#ff0000,stroke-width:2px,color:#000 style B fill:#ccffcc,stroke:#00aa00,stroke-width:2px,color:#000 style C fill:#ccccff,stroke:#0000ff,stroke-width:2px,color:#000

5. Construction of a Public-Key Cryptosystem (Regev’s Cryptosystem) using LWE

Now that we understand the difficulty of the LWE problem, let’s look at the basic public-key cryptosystem proposed by Oded Regev to see how it is used for encryption and decryption. Here, we will explain the most basic mechanism for encrypting a 1-bit message $M \in \{0, 1\}$.

5.1 Key Generation

Determine the system parameters: the modulus prime number $q$, the dimension $n$, and the number of equations $m$ ($m > n \log q$).
As a secret key, choose a vector $\mathbf{s} \in \mathbb{Z}_q^n$ at random.
Generate a random matrix $A \in \mathbb{Z}_q^{m \times n}$.
Choose a small error vector $\mathbf{e} \in \mathbb{Z}_q^m$ from an error distribution such as a discrete Gaussian distribution.
Calculate the vector $\mathbf{b} = A \mathbf{s} + \mathbf{e} \pmod q$.
The Public Key will be $(A, \mathbf{b})$.
The Secret Key will be $\mathbf{s}$.

The public key is exactly an “instance of the LWE problem.” Finding the secret key $\mathbf{s}$ from the public key $(A, \mathbf{b})$ is equivalent to solving the Search LWE problem, thereby ensuring security.

5.2 Encryption

Alice encrypts a 1-bit message $M \in \{0, 1\}$ using Bob’s public key $(A, \mathbf{b})$.

Choose a random binary vector (components are 0 or 1) $\mathbf{r} \in \{0, 1\}^m$.
As the first half of the ciphertext, compute the vector $\mathbf{u} = A^T \mathbf{r} \pmod q$. ($A^T$ is the transpose of $A$. That is, we are adding up the rows of $A$ where the component of $\mathbf{r}$ is 1).
As the second half of the ciphertext, compute the scalar $v = \mathbf{b}^T \mathbf{r} + M \cdot \lfloor \frac{q}{2} \rfloor \pmod q$. (If the message $M$ is 0, add nothing; if $1$, add exactly half the value of $q$, $\lfloor \frac{q}{2} \rfloor$).
The Ciphertext will be $(\mathbf{u}, v)$.

The intuitive meaning of encryption is to take the “sum of a random subset” for the public key matrix $A$ and vector $\mathbf{b}$. Due to the difficulty of the Decision LWE problem, this ciphertext $(\mathbf{u}, v)$ appears indistinguishable from a completely random vector and uniform random number (Semantic Security).

flowchart LR M["Message M in {0,1}"] --> Enc PK["Public Key (A, b)"] --> Enc r["Random binary vector r"] --> Enc subgraph Enc ["Encryption Process"] direction TB u_calc["u = A^T * r mod q"] v_calc["v = b^T * r + M * floor(q/2) mod q"] end Enc --> CT["Ciphertext (u, v)"]

5.3 Decryption

Bob decrypts the ciphertext $(\mathbf{u}, v)$ using the secret key $\mathbf{s}$.

Compute the following value: $D = v - \mathbf{s}^T \mathbf{u} \pmod q$
If the computed result is closer to $0$, output $M=0$; if it is closer to $\lfloor \frac{q}{2} \rfloor$, output $M=1$.

Let’s expand this mathematically to see why this can decrypt the message. Recall that $\mathbf{b} = A \mathbf{s} + \mathbf{e}$.

$$ \begin{aligned} v - \mathbf{s}^T \mathbf{u} &= (\mathbf{b}^T \mathbf{r} + M \cdot \lfloor \frac{q}{2} \rfloor) - \mathbf{s}^T (A^T \mathbf{r}) \\ &= ((A \mathbf{s} + \mathbf{e})^T \mathbf{r} + M \cdot \lfloor \frac{q}{2} \rfloor) - \mathbf{s}^T A^T \mathbf{r} \\ &= (\mathbf{s}^T A^T \mathbf{r} + \mathbf{e}^T \mathbf{r} + M \cdot \lfloor \frac{q}{2} \rfloor) - \mathbf{s}^T A^T \mathbf{r} \\ &= \mathbf{e}^T \mathbf{r} + M \cdot \lfloor \frac{q}{2} \rfloor \pmod q \end{aligned} $$

Here, $\mathbf{s}^T A^T \mathbf{r}$ perfectly cancelled out from the equation! What remains is $\mathbf{e}^T \mathbf{r} + M \cdot \lfloor \frac{q}{2} \rfloor$.

$\mathbf{e}$ is a noise vector with very small components, and $\mathbf{r}$ is a binary vector whose components are 0 or 1. Therefore, their inner product $\mathbf{e}^T \mathbf{r}$ also remains a relatively small value (if parameters are chosen properly).

If $M=0$, the result is $\mathbf{e}^T \mathbf{r}$, which is a small value close to $0$.
If $M=1$, the result is $\mathbf{e}^T \mathbf{r} + \lfloor \frac{q}{2} \rfloor$, which will be located around half the value of $q$, $\lfloor \frac{q}{2} \rfloor$.

If the parameters are designed so that the absolute value of the error $\mathbf{e}^T \mathbf{r}$ stays under $\frac{q}{4}$, Bob can accurately determine (decrypt) the message $M$ just by seeing whether the computed result is closer to $0$ or $\lfloor \frac{q}{2} \rfloor$. This is the beautiful mechanism by which LWE-based cryptography functions.

flowchart LR CT["Ciphertext (u, v)"] --> Dec SK["Secret Key s"] --> Dec subgraph Dec ["Decryption Process"] direction TB calc["Compute D = v - s^T * u mod q"] check["Check if D is closer to 0 or q/2"] end calc --> check Dec --> M_out["Recovered Message M"]

6. Toy Example of LWE Cryptography Using Specific Numerical Values

Since simply listing formulas might make it hard to get a real sense of it, let’s actually set very small numerical parameters and follow the calculations from encryption to decryption. (* In actual cryptographic systems, values of $n$ of 500 or more and $q$ of several thousands or more are used to ensure security)

[Parameter Settings]

Modulus $q = 17$ (A prime number. Therefore, values take the range from $0$ to $16$)
Dimension $n = 2$
Number of equations $m = 4$
Suppose we want to encrypt the message $M = 1$.
Message shift amount: $\lfloor \frac{q}{2} \rfloor = \lfloor \frac{17}{2} \rfloor = 8$

[1. Key Generation Phase] Bob randomly chooses the secret key $\mathbf{s}$, matrix $A$, and error vector $\mathbf{e}$.

$$ \mathbf{s} = \begin{pmatrix} 3 \\ 4 \end{pmatrix} \in \mathbb{Z}_{17}^2 $$ $$ A = \begin{pmatrix} 2 & 15 \\ 1 & 8 \\ 14 & 5 \\ 9 & 10 \end{pmatrix} \in \mathbb{Z}_{17}^{4 \times 2} $$ $$ \mathbf{e} = \begin{pmatrix} 1 \\ -1 \\ 0 \\ 2 \end{pmatrix} \equiv \begin{pmatrix} 1 \\ 16 \\ 0 \\ 2 \end{pmatrix} \pmod{17} $$

Next, calculate the public key $\mathbf{b}$.

$$ A \mathbf{s} = \begin{pmatrix} 2 & 15 \\ 1 & 8 \\ 14 & 5 \\ 9 & 10 \end{pmatrix} \begin{pmatrix} 3 \\ 4 \end{pmatrix} = \begin{pmatrix} 2\times 3 + 15\times 4 \\ 1\times 3 + 8\times 4 \\ 14\times 3 + 5\times 4 \\ 9\times 3 + 10\times 4 \end{pmatrix} = \begin{pmatrix} 6 + 60 \\ 3 + 32 \\ 42 + 20 \\ 27 + 40 \end{pmatrix} = \begin{pmatrix} 66 \\ 35 \\ 62 \\ 67 \end{pmatrix} $$

Calculate this modulo 17. (e.g., $66 = 17 \times 3 + 15$)

$$ A \mathbf{s} \pmod{17} = \begin{pmatrix} 15 \\ 1 \\ 11 \\ 16 \end{pmatrix} $$

Add the error vector $\mathbf{e}$.

$$ \mathbf{b} = A \mathbf{s} + \mathbf{e} = \begin{pmatrix} 15 \\ 1 \\ 11 \\ 16 \end{pmatrix} + \begin{pmatrix} 1 \\ 16 \\ 0 \\ 2 \end{pmatrix} = \begin{pmatrix} 16 \\ 17 \\ 11 \\ 18 \end{pmatrix} \equiv \begin{pmatrix} 16 \\ 0 \\ 11 \\ 1 \end{pmatrix} \pmod{17} $$

The public key is $A$ and $\mathbf{b} = (16, 0, 11, 1)^T$.

[2. Encryption Phase] Alice encrypts the message $M = 1$. Choose a random vector $\mathbf{r}$. Here, let $\mathbf{r} = (1, 0, 1, 0)^T$.

Calculate $\mathbf{u}$.

$$ \mathbf{u} = A^T \mathbf{r} = \begin{pmatrix} 2 & 1 & 14 & 9 \\ 15 & 8 & 5 & 10 \end{pmatrix} \begin{pmatrix} 1 \\ 0 \\ 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 2 \times 1 + 14 \times 1 \\ 15 \times 1 + 5 \times 1 \end{pmatrix} = \begin{pmatrix} 16 \\ 20 \end{pmatrix} \equiv \begin{pmatrix} 16 \\ 3 \end{pmatrix} \pmod{17} $$

Calculate $v$.

$$ \mathbf{b}^T \mathbf{r} = (16, 0, 11, 1) \begin{pmatrix} 1 \\ 0 \\ 1 \\ 0 \end{pmatrix} = 16 \times 1 + 11 \times 1 = 27 \equiv 10 \pmod{17} $$

Add the value $\lfloor 17/2 \rfloor = 8$ corresponding to the message $M=1$.

$$ v = \mathbf{b}^T \mathbf{r} + M \cdot 8 = 10 + 1 \times 8 = 18 \equiv 1 \pmod{17} $$

Alice sends the ciphertext $(\mathbf{u}, v) = \left( \begin{pmatrix} 16 \\ 3 \end{pmatrix}, 1 \right)$ to Bob.

[3. Decryption Phase] Bob, upon receiving the ciphertext, decrypts it using the secret key $\mathbf{s} = (3, 4)^T$. Calculate the decryption formula: $D = v - \mathbf{s}^T \mathbf{u} \pmod{17}$.

$$ \mathbf{s}^T \mathbf{u} = (3, 4) \begin{pmatrix} 16 \\ 3 \end{pmatrix} = 3 \times 16 + 4 \times 3 = 48 + 12 = 60 \equiv 9 \pmod{17} $$ $$ D = v - \mathbf{s}^T \mathbf{u} = 1 - 9 = -8 \pmod{17} $$

Here, in the modulo 17 world, $-8$ is equal to $9$ ($-8 + 17 = 9$). Determine whether the obtained value $D = 9$ is closer to $0$ or $8$ ($\lfloor 17/2 \rfloor$). Since $9$ is clearly closer to $8$ than to $0$, Bob correctly restored $M = 1$!

Why did it become $9$? Let’s recall the previous proof. The error part is $\mathbf{e}^T \mathbf{r} = (1, -1, 0, 2) (1, 0, 1, 0)^T = 1 \times 1 + 0 \times 1 = 1$. Therefore, the calculation result is $\mathbf{e}^T \mathbf{r} + M \cdot 8 = 1 + 8 = 9$, confirming that the theoretically expected value was calculated.

7. Evolution Towards Practical Application: Ring-LWE and Module-LWE

The Standard LWE problem explained so far has an extremely strong security proof, but it has a fatal flaw in practical use. That is, “the size of the keys becomes huge” and “the computational cost is high.”

In Standard LWE, the public key includes a huge matrix $A \in \mathbb{Z}_q^{m \times n}$. When the parameter $n$ goes up to hundreds or thousands, the size of this matrix reaches several megabytes, making it too heavy to transmit and receive every time over Internet communication protocols (such as TLS). Also, multiplying a matrix and a vector requires a computational complexity of $\mathcal{O}(n^2)$.

To solve this problem, “Ring-LWE (RLWE)” and “Module-LWE (MLWE)” were introduced, incorporating an algebraic structure called polynomial rings into the lattice.

7.1 Intuition of Ring-LWE

In Ring-LWE, vectors and matrices are replaced with elements (polynomials) over a polynomial ring $\mathcal{R}_q = \mathbb{Z}_q[X]/(X^n + 1)$. (Here, $n$ is chosen as a power of 2).

Whereas the public key of Standard LWE was a matrix $A$, Ring-LWE uses a single polynomial $a(x)$. The secret key $s(x)$ and the error $e(x)$ also become polynomials. The equation looks like this:

$$ b(x) = a(x) \cdot s(x) + e(x) \pmod q $$

Since this is polynomial multiplication, by using the “Number Theoretic Transform (NTT),” which is similar to the Fast Fourier Transform (FFT), the computational complexity can be dramatically reduced to $\mathcal{O}(n \log n)$. Furthermore, because the size of the public key shrinks from a matrix to a single polynomial, the data size is reduced to $\mathcal{O}(n)$. This brings an overwhelming advantage in communication bandwidth.

Mathematically speaking, Ring-LWE reduces to a problem on a lattice with a special symmetry called an “Ideal Lattice” rather than a general lattice.

7.2 Module-LWE and NIST Standardization (Kyber / ML-KEM)

While Ring-LWE is efficient, there was some concern that the special algebraic structure of ideal lattices might become a clue for future attacks. Therefore, “Module-LWE (MLWE)” was created to take the “best of both worlds”: the conservative security of Standard LWE and the efficiency of Ring-LWE.

Module-LWE considers small matrices and vectors whose elements are polynomials. In other words, it handles modules over a ring. Currently, “CRYSTALS-Kyber” (standardized name: ML-KEM), which NIST selected as the standard for PQC key encapsulation mechanisms (KEM), is built precisely on the difficulty of this Module-LWE problem.

8. Why is it Secure Against Quantum Computers?

Finally, let’s touch upon the core issue: “Why is lattice-based cryptography considered unbreakable even when using quantum computers?”

Shor’s algorithm, which allows quantum computers to break RSA cryptography and Elliptic Curve Cryptography, is essentially an algorithm that solves the “Hidden Subgroup Problem (HSP).” The mathematical structure (finite abelian groups) behind RSA and ECC has periodicity, and by using a specific operation of quantum algorithms called the Quantum Fourier Transform (QFT), this period (hidden subgroup) can be extracted all at once.

However, lattice problems are fundamentally different. Although lattices also have periodicity, what is required in SVP and CVP is a geometric, non-linear property such as “shortest distance” or “removal of noise.” Even if a “Quantum Fourier Transform over an abelian group” like Shor’s algorithm is applied directly, useful information that would be the answer to the lattice problem cannot be efficiently extracted. To date, no quantum algorithm that can solve SVP or LWE in polynomial time has been discovered, and it is widely believed that even with the parallel computing power of quantum computers, the only effective means is near-brute-force search (about the level of square root speedup by Grover’s algorithm).

9. Conclusion

In this article, we explained the mathematical intuition of lattice-based cryptography in detail, starting from the geometric definition of a lattice, to the formulation of the LWE problem, and the construction of a public-key cryptosystem.

A Lattice is a discrete space represented by integer-coefficient linear combinations of basis vectors, and finding a “good basis” close to orthogonal (SVP) becomes difficult in high dimensions.
The LWE (Learning With Errors) problem is the problem of solving a system of linear equations with noise, and since it is tied to the difficulty of worst-case problems on lattices, it provides a powerful security foundation.
By using the LWE problem, encryption and decryption (Regev’s Cryptosystem) are realized through an ingenious mechanism of intentionally adding and removing noise.
In real-world protocols, Ring-LWE and Module-LWE using polynomial rings are adopted to improve communication efficiency and computation speed, serving as the foundation for the NIST-standard ML-KEM.

As the unprecedented computational paradigm shift of quantum computers approaches, it is quite romantic that “lattice-based cryptography,” born from the depths of classical linear algebra and number theory, will bear the foundation of future internet security. The math that forms the foundation of lattice-based cryptography is by no means too esoteric, and anyone with a basic knowledge of linear algebra and probability can fully understand its beautiful structure. We hope this article has helped you understand lattice-based cryptography, the core of PQC.

The Riemann Hypothesis and the Distribution of Prime Numbers: A Deep Connection with Modern Cryptography

Fri, 11 Sep 2026 16:00:00 +0900

1. Introduction: The Mystery of the Universe in Prime Numbers and the Riemann Hypothesis

“Prime Numbers” are natural numbers divisible only by 1 and themselves, often called the “atoms” in the world of mathematics. The sequence of 2, 3, 5, 7, 11, 13… appears at first glance to be unordered and random. Ever since the ancient Greek mathematician Euclid proved that “there are infinitely many prime numbers,” countless mathematicians have challenged themselves to unravel the regularity hidden in the arrangement of these primes.

The closest anyone has come to the mystery of prime numbers is the “Riemann Hypothesis” proposed by the German mathematician Bernhard Riemann in 1859. The Riemann Hypothesis is one of the most important and unsolved difficult problems in modern mathematics, and carries a $1 million prize as one of the Millennium Prize Problems designated by the Clay Mathematics Institute.

At first glance, a difficult problem in pure mathematics regarding the distribution of prime numbers may seem unrelated to our daily lives. However, internet security, which supports the infrastructure of modern society, especially modern cryptographic technologies such as RSA cryptography and Elliptic Curve Cryptography (ECC), relies deeply on the properties of gigantic prime numbers.

In this article, we will embark on a mathematical journey from the distribution of prime numbers to the Prime Number Theorem, the Riemann Zeta function, and the core of the Riemann Hypothesis. We will provide an extremely detailed and deep explanation of how it is connected to modern cryptography and what would happen to the world if the Riemann Hypothesis were proven.

2. The Prime Number Theorem and the Distribution of Primes: Gauss’s Discovery

To understand how prime numbers are distributed, mathematicians considered the Prime-counting function $\pi(x)$, which represents “how many prime numbers exist up to a certain number $x$.”

For example:

$\pi(10) = 4$ (2, 3, 5, 7)
$\pi(100) = 25$
$\pi(1000) = 168$

The 15-year-old genius mathematician Carl Friedrich Gauss calculated vast tables of prime numbers and discovered that the frequency of appearance of prime numbers decreases in inverse proportion to the natural logarithm $\ln x$. That is, he conjectured that the probability of finding a prime number near a certain number $x$ is approximately $\frac{1}{\ln x}$.

Expressing this using integration gives the Logarithmic integral $\text{Li}(x)$:

$$ \text{Li}(x) = \int_{2}^{x} \frac{dt}{\ln t} $$

Gauss’s conjecture was later independently proven in 1896 by Jacques Hadamard and Charles Jean de la Vallée Poussin, and was established as the Prime Number Theorem (PNT).

$$ \lim_{x \to \infty} \frac{\pi(x)}{\text{Li}(x)} = 1 $$

Or, it can be approximately expressed as follows:

$$ \pi(x) \sim \frac{x}{\ln x} $$

Through this theorem, it became clear that prime numbers have a very smooth and predictable distribution when viewed macroscopically. However, microscopically, there is always an “error” or “fluctuation” between $\pi(x)$ and $\text{Li}(x)$. The true nature of this fluctuation is exactly the greatest mystery that the Riemann Hypothesis attempts to unravel.

3. The Riemann Zeta Function and the Euler Product

The most powerful weapon for analyzing the distribution of prime numbers is the Riemann Zeta Function. Originally, it was an infinite series defined by Leonhard Euler for real numbers $s > 1$.

$$ \zeta(s) = \sum_{n=1}^\infty \frac{1}{n^s} = 1 + \frac{1}{2^s} + \frac{1}{3^s} + \frac{1}{4^s} + \dots $$

One of Euler’s greatest achievements was proving that this infinite series can be expressed as an infinite product over all prime numbers $p$. This is the Euler Product Formula.

$$ \zeta(s) = \prod_{p \text{ prime}} \frac{1}{1 - p^{-s}} = \left( \frac{1}{1 - 2^{-s}} \right) \left( \frac{1}{1 - 3^{-s}} \right) \left( \frac{1}{1 - 5^{-s}} \right) \dots $$

An intuitive understanding of the proof is that if each term on the right side is expanded as a geometric series and multiplied together, by the Fundamental Theorem of Arithmetic (every natural number can be uniquely represented as a product of primes), the sum of the reciprocals of natural numbers on the left side is perfectly reconstructed.

This single mathematical formula became the bridge connecting analysis (infinite series, continuous functions) and number theory (prime numbers, discrete numbers). Investigating the Zeta function is synonymous with investigating the distribution of prime numbers.

4. Analytic Continuation and Extension to the Complex Plane

Riemann’s genius lay in extending the variable $s$ of $\zeta(s)$, which Euler had considered only for real numbers, to complex numbers $s = \sigma + it$ (where $\sigma$ is the real part and $t$ is the imaginary part).

The original infinite series only converges for $\sigma > 1$, but using a technique called “Analytic Continuation,” Riemann extended the definition so that $\zeta(s)$ is meaningful over the entire complex plane, excluding the pole at $s = 1$.

He further derived a beautiful functional equation satisfied by the Zeta function:

$$ \zeta(s) = 2^s \pi^{s-1} \sin\left(\frac{\pi s}{2}\right) \Gamma(1-s) \zeta(1-s) $$

Here, $\Gamma(x)$ is the Gamma function. Through this equation, the properties of the left half-plane can be known from the properties of the right half-plane.

Zeros of the Zeta Function

Complex numbers $s$ for which the value of the Zeta function becomes 0 are called “zeros”. From the functional equation, when $s$ is a negative even integer ($-2, -4, -6, \dots$), $\sin(\pi s / 2)$ becomes 0, resulting in $\zeta(s) = 0$. These are called Trivial zeros.

However, what is important in the distribution of prime numbers are the other zeros, namely the Non-trivial zeros that exist in the “Critical strip” where $0 \le \sigma \le 1$.

5. The Core of the Riemann Hypothesis and the Explicit Formula

Riemann calculated a small number of zeros and formulated a remarkable conjecture. This is the Riemann Hypothesis.

Riemann Hypothesis All non-trivial zeros of the Riemann Zeta function $\zeta(s)$ lie on the line where the real part is $1/2$ ($\text{Re}(s) = 1/2$).

This line with a real part of $1/2$ is called the “Critical line”.

graph TD A["Riemann Zeta function ζ(s)"] --> B["Extension to the complex plane by analytic continuation"] B --> C["Trivial zeros (s = -2, -4, -6 ...)"] B --> D["Non-trivial zeros (0 <= Re(s) <= 1)"] D --> E["Riemann Hypothesis"] E --> F["All non-trivial zeros lie on Re(s) = 1/2"] F --> G["To the proof of the limit of the error term in the prime distribution"]

Why is the Riemann Hypothesis so important? It is because the zeros of the Zeta function completely determine the distribution of prime numbers.

Riemann and later the mathematician von Mangoldt derived an “Explicit formula” that accurately describes the distribution of primes. Using the Chebyshev function $\psi(x)$, it is expressed as follows:

$$ \psi(x) = x - \sum_{\rho} \frac{x^\rho}{\rho} - \ln(2\pi) - \frac{1}{2}\ln(1 - x^{-2}) $$

Here, $\rho$ runs over all the non-trivial zeros of the Zeta function. The main term is $x$ (which corresponds to the Prime Number Theorem), and by adding and subtracting wave-like terms depending on the zeros $\rho$, the precise step-like distribution of primes is restored. The non-trivial zeros can be said to represent the “frequencies (waves)” of the distribution of primes.

If the Riemann Hypothesis is true, and the real part of all non-trivial zeros $\rho$ is exactly $1/2$, the error term of the Prime Number Theorem would fall within the theoretically smallest possible range.

$$ |\pi(x) - \text{Li}(x)| \le \frac{1}{8\pi} \sqrt{x} \ln x \quad \text{for} \quad x \ge 2657 $$

In other words, if the Riemann Hypothesis is true, it proves that prime numbers are distributed as “regularly and beautifully” as we can possibly imagine.

6. The Inseparable Relationship Between Modern Cryptography and Prime Numbers

So far, we have been in the realm of profound pure mathematics, but this property of prime numbers fundamentally supports modern digital society. A representative example of this is public-key cryptography, such as RSA cryptography.

The security of all communications, such as credit card payments on the internet, password transmissions, and electronic signatures in blockchains, relies on “prime numbers.”

How RSA Cryptography Works

The security of RSA cryptography is based on the mathematical fact that “factorizing a composite number with a large number of digits is extremely difficult” (the integer factorization problem).

Key Generation: Randomly select two gigantic prime numbers $p$ and $q$ (for example, 2048 bits each). Multiply them to calculate $N = p \times q$. This $N$ becomes a part of the public key. Using Euler’s totient function $\phi(N) = (p-1)(q-1)$, generate a private key $d$:
$$ e \times d \equiv 1 \pmod{\phi(N)} $$
Encryption and Decryption: The plaintext $M$ is converted into ciphertext $C$ using the public keys $e, N$:
$$ C \equiv M^e \pmod{N} $$
Only the person possessing the private key $d$ can decrypt it:
$$ M \equiv C^d \pmod{N} $$

graph LR A["Plaintext"] --> B["Encrypt with public key (e, N)"] B --> C["Ciphertext"] C --> D["Decrypt with private key (d)"] D --> E["Original Plaintext"] F["Attacker"] -- "Attempts to factorize N" --> C F -.-> G["d cannot be calculated without knowing p and q"]

To break RSA cryptography, one must find (factorize) the original primes $p$ and $q$ from a massive $N$. Even using currently mainstream algorithms (such as the General Number Field Sieve, GNFS), factorizing a number with hundreds of digits would take significantly more time than the age of the universe, even with a supercomputer.

7. The Impact of the Riemann Hypothesis on Cryptography

So, how do the “Riemann Hypothesis,” which sits at the pinnacle of pure mathematics, and “cryptography” intersect?

7.1. Prime Generation Algorithms (Primality Testing) and the Generalized Riemann Hypothesis (GRH)

To operate RSA cryptography, gigantic prime numbers $p$ and $q$ must first be generated. However, it is not easy to reliably and quickly determine whether “a certain number is prime.”

Currently, what is used practically is a probabilistic algorithm called the Miller-Rabin primality test. Although this algorithm is fast, there is a risk of “pseudoprimes,” where a composite number is incorrectly identified as a prime number with an extremely low probability.

However, if we assume the “Generalized Riemann Hypothesis (GRH)”, which extends the Riemann Hypothesis to Dirichlet L-functions, to be true, the story changes dramatically. If the GRH is true, an upper bound on the number of tests in the Miller-Rabin test is mathematically guaranteed, and it is elevated from a probabilistic algorithm to a “deterministic polynomial-time algorithm” (this was a significant fact known even before the discovery of the AKS primality test).

In short, the Riemann Hypothesis (and its generalization) plays a role in directly validating the foundation of cryptography: “Can we generate massive prime numbers quickly and with absolute confidence?”

7.2. Relationship with Factorization Algorithms

When evaluating the computational complexity of algorithms for breaking codes (such as the General Number Field Sieve), knowledge of the distribution of prime numbers is also essential. Many integer factorization algorithms rely on the distribution of “Smooth numbers” (numbers that only have small prime factors).

To strictly evaluate how frequently smooth numbers appear, a deep understanding of the distribution of primes is necessary, and analytic number theory techniques directly connected to the Zeta function and the Riemann Hypothesis are fully utilized here as well. If the Riemann Hypothesis is proven and the error in prime distribution is completely determined, it will become possible to more accurately assess the performance limits of integer factorization algorithms.

8. Will Cryptography be Broken if the Riemann Hypothesis is Proven?

There is an urban legend that says, “If the Riemann Hypothesis is solved, RSA cryptography will instantly collapse,” but this is mathematically inaccurate.

The proof of the Riemann Hypothesis itself would not immediately yield a magical algorithm that drastically speeds up integer factorization. This is because the Riemann Hypothesis is ultimately a theorem about the “macroscopic regularity of the distribution” of primes, and it does not directly tell us which primes divide an individual number $N$ (a local property).

However, the impact is not zero. This is because, in the process of proving the Riemann Hypothesis, it is extremely likely that “new mathematical tools” and “unknown analytical methods” will be discovered. Looking at history, when Fermat’s Last Theorem or the Poincaré Conjecture were proven, the new theories developed during the process greatly advanced the entirety of mathematics.

If unknown algebraic geometry methods or non-commutative geometry methods are established that can completely manipulate the properties of the zeros of the Riemann Zeta function, it cannot be denied that this might ultimately lead to the discovery of a groundbreaking integer factorization algorithm (for example, a classical algorithm that reduces the computational complexity to polynomial time). In that sense, cryptographers can never take their eyes off the developments surrounding the Riemann Hypothesis.

Quantum Computers and Shor’s Algorithm

A more direct and realistic threat to cryptography is not the proof of the Riemann Hypothesis, but quantum computers. “Shor’s Algorithm,” published by Peter Shor in 1994, proved that integer factorization can be solved in polynomial time if a quantum computer with sufficient performance exists. Consequently, RSA cryptography and Elliptic Curve Cryptography will be fundamentally broken.

Currently, a transition to “Post-Quantum Cryptography (PQC)” (such as lattice-based cryptography), which cannot be decrypted even by quantum computers, is progressing worldwide. Cryptography relying on prime numbers may be coming to the end of its golden age in a sense, but the mathematical value of prime numbers themselves will never be lost.

9. Conclusion: The Intersection of Mathematical Abstraction and Real Society

graph TD A["Exploration of pure mathematics"] --> B["Elucidation of the Riemann Hypothesis"] B --> C["Complete understanding of prime distribution"] C --> D["Leap in the development of number theory and algebraic geometry"] D -.-> E["Possibility of new integer factorization algorithms"] E -.-> F["Update of security evaluations for cryptography"] A --> G["Applied mathematics and computer science"] G --> H["Efficiency in primality testing and cryptographic generation"] H --> F

The insatiable exploration of prime numbers that has continued since ancient Greece was elevated by the genius of Riemann into a beautiful symphony (the zeros of the Zeta function) on the complex plane. Amazingly, after several centuries, the crystallization of that purely innocent mathematics is being applied as the strongest shield ensuring the safety of the internet society.

The Riemann Hypothesis symbolizes both the “abstract beauty” of mathematics and its “astonishing applicability to the physical world and real society.”

When this massive mountain of mathematics, whose peak no one has yet reached, is conquered someday, we will completely understand the universal truth of prime numbers and gain a new perspective on the foundation of our information society. Studying cryptography is, in itself, a journey tracing the history of human wisdom.

Fast Primality Testing Algorithm Implemented in C++ (Miller-Rabin, etc.)

Fri, 11 Sep 2026 14:00:00 +0900

Introduction: Why Do We Need Fast Primality Testing?

In the worlds of computer science, cryptography, and competitive programming, determining whether a given number is prime quickly and accurately is an extremely important and fundamental task. For example, public-key cryptography such as RSA, which underpins the security of modern internet society, relies on the generation of massive prime numbers and the difficulty of factoring their product as the basis of its security. Therefore, it is no exaggeration to say that the technology to instantly identify whether a massive number is prime is a technology that supports the foundation of digital society.

Also, in competitive programming (such as AtCoder and Codeforces), primality testing is a frequently occurring theme. For massive inputs with constraints like $N \le 10^{18}$, in situations where you need to perform tens of thousands of primality tests within 1 second, traditional, naive algorithms will certainly result in a Time Limit Exceeded (TLE).

In this article, we will thoroughly explain everything from naive primality testing algorithms to the probabilistic primality testing method “Fermat’s Primality Test,” and the practically strongest fast algorithm that overcomes its weaknesses: the “Miller-Rabin Primality Test.” We will cover everything from the mathematical background to a highly optimized implementation in C++. In particular, for 64-bit integers ($N < 2^{64}$), we will explain in detail a method that goes beyond probabilistic testing to “100% deterministic primality testing” and provide C++ source code that you can use directly in practice.

1. The Basics of Primality Testing and Trial Division

A prime number is a natural number greater than 1 that has no positive divisors other than 1 and itself. According strictly to the definition of a prime number, to determine whether a given integer $N$ is prime, you can try dividing $N$ by all integers from $2$ to $N-1$. If it is never divisible, it is a prime number; if it is divisible even once, it is a composite number (not a prime).

However, the time complexity of this method is $O(N)$. When $N$ is a massive number like $10^{18}$, even modern computers would take an enormous amount of time to calculate it.

Optimizing Trial Division: Searching up to $\sqrt{N}$

When a composite number $N$ is expressed as $a \times b = N$ ($a \le b$), it is always true that $a \le \sqrt{N}$. Therefore, the primality testing loop does not need to run up to $N-1$; it is sufficient to check up to $\sqrt{N}$.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


#include <iostream>

// Primality testing by trial division (O(sqrt(N)))
bool is_prime_trial_division(long long n) {
 if (n <= 1) return false;
 if (n == 2 || n == 3) return true;
 if (n % 2 == 0) return false;

 // Only check odd numbers from 3 onwards
 for (long long i = 3; i * i <= n; i += 2) {
 if (n % i == 0) return false;
 }
 return true;
}

The time complexity of this algorithm is $O(\sqrt{N})$. If $N \le 10^{12}$, it can be computed instantly, but when $N \approx 10^{18}$, the number of loop iterations is about $10^9$. Even with a C++ runtime, this takes hundreds of milliseconds to several seconds, making it unsuitable for multiple tests.

2. Fermat’s Primality Test: The Dawn of Probabilistic Primality Testing

To break through the limitations of trial division, “Probabilistic Algorithms” using theorems from number theory were devised. A prime example is the “Fermat Primality Test,” which utilizes Fermat’s Little Theorem.

Fermat’s Little Theorem

This theorem, discovered by Pierre de Fermat, states the following:

For any prime $p$ and any integer $a$ that is coprime to $p$ (not a multiple of $p$), the following congruence holds:
$$ a^{p-1} \equiv 1 \pmod p $$

Taking the contrapositive of this theorem, we can say, “For a given integer $N$ and an integer $a$ coprime to $N$, if $a^{N-1} \not\equiv 1 \pmod N$, then $N$ is definitely a composite number.” The Fermat primality test uses this property by choosing a random base $a$ for the number $N$ to be tested, calculating $a^{N-1} \pmod N$, and checking if it equals $1$.

Fast Modular Exponentiation (Binary Exponentiation)

To perform the Fermat test, we need to quickly compute the massive exponentiation $a^{N-1} \pmod N$. We use “Modular Exponentiation / Binary Exponentiation” for this. The time complexity is $O(\log N)$, making it extremely fast.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


// Calculating a^b mod m using binary exponentiation
long long mod_pow(long long a, long long b, long long m) {
 long long res = 1;
 a %= m;
 while (b > 0) {
 if (b & 1) res = (__int128_t)res * a % m;
 a = (__int128_t)a * a % m;
 b >>= 1;
 }
 return res;
}

Note: To prevent overflow here, we use __int128_t (a 128-bit integer), which is an extension in GCC/Clang, to hold intermediate products.

Pseudoprimes and Carmichael Numbers

The Fermat test is very powerful, but it has a fatal flaw. There are numbers $N$ that are composite, yet $a^{N-1} \equiv 1 \pmod N$ holds for all $a$ (where $a$ is coprime to $N$).

Such numbers are called “absolute pseudoprimes” or “Carmichael numbers.” The smallest Carmichael number is $561 = 3 \times 11 \times 17$. Because Carmichael numbers exist, the Fermat test alone cannot provide a “100% probability” deterministic test. No matter how many different $a$’s you try, numbers like $561$ will always pretend to be prime (fooling the test).

3. Miller-Rabin Primality Test

The “Miller-Rabin Primality Test,” devised by Gary L. Miller and Michael O. Rabin, brilliantly overcame the weakness of the Fermat test (the existence of Carmichael numbers). Currently, it is the most widely used practical fast primality testing algorithm in internal libraries of various programming languages and in key generation for cryptographic systems.

Mathematical Principles

The Miller-Rabin algorithm uses Fermat’s Little Theorem along with the property that “in a residue field modulo a prime ($\mathbb{Z}/p\mathbb{Z}$), the only solutions to $x^2 \equiv 1 \pmod p$ are $x \equiv 1$ or $x \equiv -1$” (if modulo a composite number, other non-trivial square roots may exist).

Subtracting $1$ from the odd number $N$ you want to test always results in an even number, $N-1$. We divide $N-1$ by $2$ as many times as possible and express it in the following form:

$$ N-1 = d \cdot 2^s $$

(Where $d$ is odd and $s \ge 1$)

For any base $a$ ($1 < a < N-1$), we verify whether $a^{N-1} \equiv 1 \pmod N$ according to Fermat’s Little Theorem, but we perform the calculation in steps. Specifically, we repeatedly square it in the order of $a^d, a^{d \cdot 2}, a^{d \cdot 4}, \ldots, a^{d \cdot 2^s}$.

The conditions for the Miller-Rabin test to determine that $N$ is “prime (or probably prime with a high probability)” are that either of the following holds true:

$a^d \equiv 1 \pmod N$
There exists some $r$ ($0 \le r < s$) such that $a^{d \cdot 2^r} \equiv -1 \pmod N$.
- Note: In C++ modulo arithmetic, $-1 \pmod N$ is $N-1$.

If $N$ is a prime number, this condition will absolutely be satisfied for any $a$. Conversely, it has been mathematically proven that if $N$ is a composite number, the probability of satisfying this condition (the probability of being fooled) when a random $a$ is chosen is $\frac{1}{4}$ or less. If we perform $k$ independent tests, the probability of a false positive becomes $\left(\frac{1}{4}\right)^k$ or less, which can practically be considered zero. There are no numbers like Carmichael numbers that can “absolutely deceive” the test.

Algorithm Flow of the Miller-Rabin Method (Mermaid Flowchart)

The following diagram shows the logical flow of a single Miller-Rabin primality test (a test for a single base $a$).

graph TD Start["Start Test (Input: N, a)"] --> CalcDS["Calculate d (odd) and s such that N-1 = d * 2^s"] CalcDS --> CalcX["Calculate x = a^d mod N"] CalcX --> CheckX1{"x == 1 or x == N-1 ?"} CheckX1 -- "Yes" --> ReturnTrue["Probably Prime"] CheckX1 -- "No" --> LoopStart["Start loop for r = 1 to s-1"] LoopStart --> LoopCondition{"r < s ?"} LoopCondition -- "No" --> ReturnFalse["Definitely Composite"] LoopCondition -- "Yes" --> SquareX["Calculate x = (x * x) mod N"] SquareX --> CheckXMinus1{"x == N - 1 ?"} CheckXMinus1 -- "Yes" --> ReturnTrue CheckXMinus1 -- "No" --> CheckXOne{"x == 1 ?"} CheckXOne -- "Yes" --> ReturnFalse CheckXOne -- "No" --> LoopNext["Increment r by 1 and continue"] LoopNext --> LoopCondition

4. Deterministic Testing for 64-bit Integers

The Miller-Rabin primality test is inherently a “probabilistic” algorithm, but if the upper bound of $N$ is fixed, we can perform a “100% deterministic” primality test by trying a specific set of multiple $a$’s (bases). This is called the Deterministic Miller-Rabin Test.

Research by Jim Sinclair and others has shown that for all integers $N < 2^{64}$ (about $1.8 \times 10^{19}$), testing with the following $7$ prime numbers as bases $a$ is sufficient for a completely deterministic test.

List of bases $a$ to test: {2, 325, 9375, 28178, 450775, 9780504, 1795265022}

Alternatively, another well-known set uses the following $12$ prime numbers, which can also perfectly test up to $N < 2^{64}$. {2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37}

This time, to enhance the simplicity and reliability of the algorithm, we will adopt the method based on the latter $12$ primes (or the more optimized $7$ bases). In our C++ implementation, we will optimize by dividing the range with conditional branches to minimize the number of tests.

5. Highly Optimized C++ Implementation

Now, let’s synthesize the mathematical theory and algorithm design up to this point, and present the implementation code for a Miller-Rabin primality testing function that is at the highest level of performance in modern C++.

Key Implementation Points

Avoiding Overflow in 64-bit Integer Multiplication: When $N \approx 10^{18}$, $x \times x$ in modular multiplication can be up to $10^{36}$, easily overflowing the maximum value of a standard 64-bit integer (uint64_t or long long), which is $1.8 \times 10^{19}$. To solve this problem, we use the __int128_t (or unsigned __int128) extension type available in GCC and Clang to perform calculations with 128-bit precision before applying the modulo. This allows for fast modular multiplication without needing complex algorithms.
Selecting Deterministic Bases: When the value of $N$ is small, we optimize it so that we only need to test a few bases.

Complete C++ Source Code

Below is the complete source code ready for practical use. You can copy and use this code directly in environments like competitive programming.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96


#include <iostream>
#include <vector>
#include <cstdint>
#include <initializer_list>

using namespace std;

// Fast (a * b) mod m using 128-bit integers
inline uint64_t mod_mul(uint64_t a, uint64_t b, uint64_t m) {
 return (uint64_t)((unsigned __int128)a * b % m);
}

// Calculate (base^exp) mod m using binary exponentiation
uint64_t mod_pow(uint64_t base, uint64_t exp, uint64_t m) {
 uint64_t res = 1;
 base %= m;
 while (exp > 0) {
 if (exp & 1) res = mod_mul(res, base, m);
 base = mod_mul(base, base, m);
 exp >>= 1;
 }
 return res;
}

// Deterministic 64-bit integer test using Miller-Rabin primality test
bool is_prime_miller_rabin(uint64_t n) {
 // Pre-checking boundary values and small primes
 if (n < 2) return false;
 if (n == 2 || n == 3 || n == 5 || n == 7) return true;
 if (n % 2 == 0 || n % 3 == 0 || n % 5 == 0 || n % 7 == 0) return false;

 // Decompose into the form n-1 = d * 2^s
 uint64_t d = n - 1;
 int s = 0;
 while ((d & 1) == 0) {
 d >>= 1;
 s++;
 }

 // List of bases to use for testing
 // Optimization to minimize the number of bases tested depending on the size of N
 vector<uint64_t> bases;
 if (n < 4759123141ULL) {
 bases = {2, 7, 61};
 } else if (n < 1122004669633ULL) {
 bases = {2, 13, 23, 1662803};
 } else {
 // 7 bases that make it deterministic for all numbers N < 2^64
 bases = {2, 325, 9375, 28178, 450775, 9780504, 1795265022};
 }

 // Run the test for each base
 for (uint64_t a : bases) {
 a %= n;
 if (a == 0) continue; // If a is a multiple of n, it is untestable but not a prime

 uint64_t x = mod_pow(a, d, n);
 if (x == 1 || x == n - 1) continue; // Passed first condition, move to next base

 bool composite = true;
 // Loop s-1 times (x = x^2 mod n)
 for (int r = 1; r < s; r++) {
 x = mod_mul(x, x, n);
 if (x == n - 1) {
 composite = false; // Passed second condition, possibly prime
 break;
 }
 }

 // If it doesn't satisfy any condition, it is definitely a composite number
 if (composite) return false;
 }

 // If it passes the conditions for all bases, it is definitely prime
 return true;
}

int main() {
 // Test samples
 vector<uint64_t> test_cases = {
 1000000007, // Famous prime number
 998244353, // Famous prime number
 1000000000000000003, // Prime number around 10^18
 1000000000000000007, // Composite number (10^18 + 7)
 561, // Carmichael number (composite number)
 18446744073709551557ULL // One of the largest prime numbers near 2^64
 };

 for (uint64_t n : test_cases) {
 cout << n << " is "
 << (is_prime_miller_rabin(n) ? "Prime" : "Composite")
 << endl;
 }

 return 0;
}

6. Algorithm Complexity and Performance Evaluation

Let’s discuss the performance of the implemented algorithm.

Time Complexity

Trial Division: $O(\sqrt{N})$
Fermat Test: Exponentiation calculation $O(\log N) \times k$ (where $k$ is the number of trials)
Miller-Rabin Method: Exponentiation calculation and looping $O(\log N) \times k$

In a 64-bit environment ($N \le 2^{64}$), the deterministic Miller-Rabin method above verifies at most $7$ bases. Therefore, we can treat $k \le 7$ as a constant, and the overall time complexity is strictly $O(\log N)$. Even in the maximum case ($N \approx 10^{19}$), the number of execution steps is at most $7 \times 64 = 448$ basic operations, and the execution time is less than a few microseconds ($10^{-6}$ seconds). Compared to the $O(\sqrt{N})$ of trial division (loop iterations $\approx 4 \times 10^9$), a speedup of several million times is achieved.

Further Optimization: Montgomery Multiplication

In the implementation of this article, division (modulo operation %) is performed using the 128-bit integer extension type __int128_t. Even on modern CPUs, integer division (DIV instruction) is an expensive instruction that takes tens of cycles compared to addition or multiplication.

Library creators and competitive programmers seeking ultimate optimization sometimes adopt a method called Montgomery Multiplication. Montgomery multiplication is an astonishing algorithm that maps numbers to a special “Montgomery space,” replacing expensive modulo operations (divisions) with “only bit shifts and multiplications.” By incorporating this into the modular multiplication of the Miller-Rabin test, the execution speed can be further increased by a factor of 2 to 3. This is a very deep topic, so I would like to explain it in detail in another article.

7. Conclusion

In this article, we covered everything from the basics of primality testing to advanced topics all at once. Let’s review the key points.

Trial Division is reliable, but its time complexity of $O(\sqrt{N})$ makes it impractical when $N$ exceeds $10^{12}$.
Fermat’s Primality Test is extremely fast at $O(\log N)$, but it has a fatal flaw of being fooled by absolute pseudoprimes like Carmichael numbers.
The Miller-Rabin Primality Test is a practical and powerful algorithm that eliminates the weaknesses of the Fermat test.
In the C++ implementation, leveraging __int128_t allows for safe handling of 64-bit integer multiplication overflows.
Within the range of 64-bit integers ($N < 2^{64}$), choosing $7$ or $12$ specific primes as bases makes it possible to perform deterministic (100% accurate) primality testing, rather than probabilistic.

Fast primality testing is an unavoidable technology in calculations dealing with massive numbers. The C++ Miller-Rabin source code provided in this article is robust enough to be used directly in practice. Please try using it in your own projects and algorithmic competitions.

graph LR TrialDivision["Trial Division (O(√N))"] --> Fermat["Fermat Test (O(log N), has flaws)"] Fermat --> MillerRabin["Miller-Rabin Method (O(log N), can be deterministic)"] MillerRabin --> Montgomery["+ Montgomery Multiplication (Constant factor speedup)"] style MillerRabin fill:#f9f,stroke:#333,stroke-width:2px

The world of algorithms, where programming and mathematics intersect, is incredibly beautiful and profound. I hope this helps you in your future studies.

Reference:

Pomerance, C., Selfridge, J. L., & Wagstaff, S. S. (1980). The pseudoprimes to 25.10^9. Mathematics of Computation.
Sinclair, J. (2011). Deterministic Miller-Rabin primality testing.

[Math Explanation] Explaining How RSA Encryption Works So Even a High Schooler Can Understand

Fri, 11 Sep 2026 13:00:00 +0900

One of the technologies supporting the safety of our internet society from the ground up is “RSA Encryption”. Many of the communications we casually use every day, such as credit card payments in online shopping, SNS exchanges with friends, and the transmission/reception of company confidential information, are protected by this RSA encryption and its successor technologies.

However, when you hear the word “cryptography”, you might imagine complex cipher machines like those in spy movies, or super advanced mathematics that only a few geniuses can understand. It is true that modern cryptographic theory is based on advanced mathematics, but the fundamental mechanism of RSA encryption can be fully understood if you have the knowledge of high school mathematics (properties of integers, prime numbers, congruences, etc.).

In this article, taking high school math knowledge as a starting point, I will thoroughly explain step-by-step the mathematical principles by which RSA encryption operates and why it is so difficult to crack. I will explain carefully with concrete examples so that even those who are not very good at math can understand.

1. Symmetric-key and Public-key Cryptography

Before getting into the mathematical mechanisms of RSA encryption, let’s first organize the basic ideas of cryptography. Cryptographic methods can be broadly divided into two types: “Symmetric-key Cryptography” and “Public-key Cryptography”.

1.1 Limitations of Symmetric-key Cryptography

Many of the traditionally used ciphers are what is called the “Symmetric-key Cryptography” method. This is a method that uses the same key for both “encryption (converting a message into secret ciphertext)” and “decryption (restoring the ciphertext back to the original message)”.

For example, suppose Alice sends a secret letter to Bob. Alice uses a padlock (symmetric key) to put the letter in a box and lock it. In order for Bob to open that box, he needs to have the exact same key that Alice used.

There is a major problem with this method. It is the “key distribution problem”. When Alice and Bob, who are far apart, communicate for the first time, how should they share the key without it being eavesdropped on? If the key is stolen by a third party while being mailed, all subsequent encrypted communications will be completely leaked.

1.2 The Breakthrough Invention: “Public-key Cryptography”

“Public-key Cryptography” was invented to solve this key distribution problem. RSA encryption is also one of this kind.

In public-key cryptography, we use two different keys: a “key for encryption (public key)” and a “key for decryption (private key)”.

The receiver, Bob, creates a pair of “public key” and “private key”.
Bob publishes the “public key” to the world (it doesn’t matter who gets it).
The sender, Alice, encrypts her message using Bob’s “public key” and sends it.
The encrypted message can only be decrypted with the “private key” that only Bob possesses.

Comparing this to padlocks, Bob makes many “open padlocks (public keys)” and scatters them all over the world. Alice puts her message addressed to Bob into a box and snaps it shut using a padlock of Bob’s that she picked up. Once the padlock is closed, it can only be opened with the “master key (private key)” that Bob holds. Even if someone steals the box on the way, they cannot open it because they don’t have the master key.

graph TD A["Alice (Sender)"] --> B["Plaintext (Message)"] B --> C["Encryption Process"] D["Bob's Public Key (Available to anyone)"] --> C C --> E["Send via Internet: Ciphertext"] E --> F["Decryption Process"] G["Bob's Private Key (Held only by Bob)"] --> F F --> H["Restored Plaintext (Message)"] H --> I["Bob (Receiver)"]

In order to realize this epoch-making system, a kind of “one-way function (a one-way mathematical puzzle)” is necessary, one where “encryption with the public key is easy, but decryption without the private key is absolutely impossible.” What was focused on as a component of that puzzle was the “prime numbers” we all know so well.

2. The Mathematical Foundation Supporting RSA 1: Prime Numbers and Prime Factorization

The security of RSA encryption is based on the mathematical fact that “prime factorization of huge numbers is extremely difficult.”

2.1 What are Prime Numbers?

A prime number is “a natural number greater than 1 that can only be divided by 1 and itself.” Example: $2, 3, 5, 7, 11, 13, 17, 19, 23...$

Prime numbers are like the “atoms” of all integers. Any natural number can be broken down into the product of prime numbers. This is called prime factorization. For example, it is known as the “Fundamental Theorem of Arithmetic” that a number can be uniquely factorized into primes (ignoring order), such as $60 = 2^2 \times 3 \times 5$.

2.2 The Difficulty of Prime Factorization (One-way Function)

What’s important here is the asymmetry that “multiplication is easy, but prime factorization is difficult.”

For example, try doing mental arithmetic for the multiplication of the following two prime numbers. $11 \times 13 = ?$ This is easy. The answer is $143$.

Then, how about the following number? Please prime factorize $323$. How is it? It should take a little time. (The answer is $17 \times 19$).

If the numbers are small, humans can manage to calculate them, but as the numbers get larger, it becomes explosively difficult to calculate even using computers. In mainstream RSA encryption today, we use a number $N = p \times q$, which is the product of two incredibly huge prime numbers $p$ and $q$ of 2048 bits (about 600 digits in decimal).

Given two huge prime numbers $p$ and $q$, it takes an instant (less than a millisecond) for a computer to calculate $N$. However, conversely, given only $N$, finding the original $p$ and $q$ takes so much time that even the current fastest supercomputer could not solve it if it ran for trillions of years.

This “computational asymmetry (one way is easy, the reverse is difficult)” is the foundation that creates the relationship between the public key and the private key.

3. The Mathematical Foundation Supporting RSA 2: Congruence (Modulo Arithmetic)

The calculations for RSA encryption are not done with addition or multiplication where numbers grow infinitely like we normally use, but rather in a world of “remainders” after dividing by a certain number. This is called congruence (modulo arithmetic).

3.1 Clock Math

Modulo arithmetic is often compared to “clock math”. If it is currently 10 o’clock, what time will it be 5 hours from now? $10 + 5 = 15$ o’clock, but on a normal 12-hour clock, we answer “3 o’clock”. This is because the remainder of 15 divided by 12 is 3.

In the world of mathematics, this is written as follows:

$$ 15 \equiv 3 \pmod{12} $$

You read this as “15 is congruent to 3 modulo 12 (the remainder when divided by 12 is equal).”

3.2 Basic Properties of Congruences

Congruences have very convenient properties very similar to equalities ($=$). Let the modulus (the divisor) be $N$. When $a \equiv b \pmod N$ and $c \equiv d \pmod N$, the following hold true:

Addition: $a + c \equiv b + d \pmod N$
Subtraction: $a - c \equiv b - d \pmod N$
Multiplication: $a \times c \equiv b \times d \pmod N$
Exponentiation: $a^k \equiv b^k \pmod N$ ($k$ is a natural number)

Particularly important is the property of “exponentiation”. This means that “the power of a remainder is equal to the remainder of the power”. For example, suppose you want to find the remainder of $7^{100}$ divided by $5$. Multiplying $7$ a hundred times and then dividing by $5$ in earnest is very hard, but if you use the properties of congruences, since $7 \equiv 2 \pmod 5$, it becomes $7^{100} \equiv 2^{100} \pmod 5$, and you can drastically simplify the calculation. This property is indispensable in the world of cryptography because we deal with exponentiations of very large numbers.

4. The Mathematical Foundation Supporting RSA 3: Euler’s Totient Function and Euler’s Theorem

From here on is the magic mathematics that forms the core of RSA encryption. “Euler’s Theorem”, a generalization of “Fermat’s Little Theorem”, makes its appearance.

4.1 Euler’s Totient Function $\phi(N)$

For a given natural number $N$, Euler’s totient function (the $\phi$ function) is a function that returns “the number of natural numbers from 1 to $N$ that are coprime with $N$ (meaning their greatest common divisor is 1).”

Let’s look at some examples.

$\phi(5)$: Among 1, 2, 3, 4, 5, the numbers coprime with 5 are 1, 2, 3, 4, which is 4 numbers. Therefore, $\phi(5) = 4$.
$\phi(6)$: Among 1, 2, 3, 4, 5, 6, the numbers coprime with 6 are 1, 5, which is 2 numbers. Therefore, $\phi(6) = 2$.

[Special Property in the Case of Prime Numbers] If $p$ is a prime number, all numbers from 1 to $p-1$ are coprime with $p$. Therefore,

$$ \phi(p) = p - 1 $$

[Special Property in the Case of the Product of Prime Numbers] For two distinct prime numbers $p$ and $q$, if $N = p \times q$, $\phi(N)$ can be easily calculated as follows.

$$ \phi(N) = \phi(p) \times \phi(q) = (p - 1)(q - 1) $$

This property functions as the “secret backdoor (trapdoor)” in RSA encryption. The person who knows $p$ and $q$ (the creator of the key) can calculate $\phi(N)$ instantly, but a third party who only knows $N$ cannot determine $\phi(N)$ unless they prime factorize $N$.

4.2 Euler’s Theorem

Leonhard Euler used this $\phi(N)$ to prove the following beautiful theorem.

Euler’s Theorem: When the integer $a$ and $N$ are coprime, the following congruence holds.

$$ a^{\phi(N)} \equiv 1 \pmod N $$

This is an amazing property that says, “If you multiply a number $a$ by itself $\phi(N)$ times and divide by $N$, the remainder will always be $1$.” (When $N$ is a prime number $p$, it becomes $a^{p-1} \equiv 1 \pmod p$, which is called Fermat’s Little Theorem).

Let’s modify this Euler’s Theorem. Multiply both sides by $a$ one more time.

$$ a^{\phi(N) + 1} \equiv a \pmod N $$

Furthermore, for any integer $k$, since $a^{k \cdot \phi(N)}$ also becomes $1^k = 1$, the following equation holds.

$$ a^{k \cdot \phi(N) + 1} \equiv a \pmod N $$

This very equation is the fundamental principle that makes the magic of RSA encryption work: “If you encrypt and then decrypt, it returns to the original.”

5. The RSA Algorithm: Steps for Key Generation, Encryption, and Decryption

Now that we have the basic knowledge, let’s finally look at the specific steps of RSA encryption. RSA encryption is roughly divided into three phases: “1. Key Generation,” “2. Encryption,” and “3. Decryption.”

flowchart TD A1["1. Choose prime numbers p, q"] --> A2["Calculate N = p × q"] A1 --> A3["Calculate φ(N) = (p-1)(q-1)"] A3 --> A4["Choose e coprime with φ(N)"] A3 --> A5["Calculate d such that e × d ≡ 1 (mod φ(N))"] A2 --> A6["Public Key (N, e)"] A4 --> A6 A5 --> A7["Private Key d"] B1["2. Plaintext message M"] --> B2["Calculate C ≡ M^e (mod N)"] A6 -.-> B2 B2 --> B3["Send ciphertext C"] B3 --> C1["3. Received ciphertext C"] C1 --> C2["Calculate M ≡ C^d (mod N)"] A7 -.-> C2 C2 --> C3["Obtain original plaintext message M"]

5.1 Key Generation

The receiver, Bob, generates a “public key” and a “private key” for himself.

Choice of Primes: Randomly choose two large prime numbers $p$ and $q$.
Calculation of Modulus $N$: Calculate $N = p \times q$. This $N$ is made public.
Calculation of $\phi(N)$: Calculate Euler’s function $\phi(N) = (p - 1)(q - 1)$. This is a secret number only Bob knows.
Choice of Public Key $e$: Choose an integer $e$ such that $1 < e < \phi(N)$ and $e$ is coprime with $\phi(N)$.
Calculation of Private Key $d$: Find an integer $d$ that satisfies the following condition. $$ e \times d \equiv 1 \pmod{\phi(N)} $$ In other words, this is “a number $d$ such that the remainder is $1$ when $e \times d$ is divided by $\phi(N)$.”

Now the key preparation is complete.

Public Key: The pair $(N, e)$. It is published to the whole world.
Private Key: $d$. Never tell anyone under any circumstances.

5.2 Encryption

Suppose Alice wants to send a secret message $M$ to Bob. (Let $M$ be a number formed by digitizing characters, and assume $0 \le M < N$). Alice calculates as follows using Bob’s public key $(N, e)$.

$$ C \equiv M^e \pmod N $$

She calculates “the remainder $C$ when message $M$ is raised to the power of $e$, divided by $N$”. This $C$ is the ciphertext.

5.3 Decryption

Bob receives the ciphertext $C$. Bob calculates as follows using his private key $d$.

$$ M \equiv C^d \pmod N $$

By calculating “the remainder when ciphertext $C$ is raised to the power of $d$, divided by $N$”, amazingly, the original message $M$ is restored!

6. Why Does Decryption Revert it to the Original? (Mathematical Proof)

You might be wondering, “Why does simply raising $C$ to the power of $d$ return it to the original $M$?” This is where the aforementioned “Euler’s Theorem” demonstrates its power.

Let’s substitute the encryption formula $C = M^e$ into the decryption calculation formula $C^d \pmod N$.

$$ C^d \equiv (M^e)^d \equiv M^{ed} \pmod N $$

Here, recall Step 5 of the key generation. When Bob made $d$, he chose it so that $e \times d \equiv 1 \pmod{\phi(N)}$. This means “the number $ed$ is a multiple of $\phi(N)$ plus $1$.” Using an integer $k$, it can be written as follows:

$$ ed = k \cdot \phi(N) + 1 $$

Substitute this into the exponent part, and decompose it using exponent rules.

$$ M^{ed} = M^{k \cdot \phi(N) + 1} = M^{k \cdot \phi(N)} \times M^1 = (M^{\phi(N)})^k \times M $$

Here, assuming that the message $M$ and $N$ are coprime, from Euler’s Theorem, we get $M^{\phi(N)} \equiv 1 \pmod N$.

$$ (M^{\phi(N)})^k \times M \equiv 1^k \times M \equiv M \pmod N $$

Therefore, the following formula beautifully holds true.

$$ C^d \equiv M \pmod N $$

Alice does not know $d$, and an eavesdropper does not know $d$ either, so only Bob, who has $d$, can extract $M$ from $C$.

7. Concrete Example: Experiencing RSA by Hand Calculation Using Small Primes

Let’s actually try encrypted communication from Alice to Bob using small numbers (prime numbers).

[Bob’s Key Generation Phase]

Choose two prime numbers $p=11$, $q=13$.
Calculate $N = 11 \times 13 = 143$.
Calculate $\phi(N) = (11 - 1) \times (13 - 1) = 10 \times 12 = 120$.
Choose a public key $e$ that is coprime with $\phi(N)=120$. Here we will use $e=7$.
Find the private key $d$. Look for $d$ such that $7 \times d \equiv 1 \pmod{120}$. In the equation $7d = 120k + 1$, when $k=6$, it becomes $721$, and $721 \div 7 = 103$. Therefore, $d = 103$.

Public Key: $(N=143, e=7)$
Private Key: $d=103$

[Alice’s Encryption Phase] Suppose she wants to send the message $M = 9$. Formula: $C \equiv 9^7 \pmod{143}$ $9^7 = 4,782,969$. Dividing this by 143 gives $33447$ with a remainder of $48$. The ciphertext became $C = 48$.

[Bob’s Decryption Phase] Bob receives the ciphertext $C = 48$ and decrypts it using his private key $d = 103$. Formula: $M \equiv 48^{103} \pmod{143}$ If you run (48 ** 103) % 143 on a calculator, the result wonderfully turns out to be “9”! The original message was successfully received.

8. How to Find the Private Key $d$: Extended Euclidean Algorithm

In the hand calculation example, we found $d=103$ by guessing to find $k$, but this method is impossible when the numbers are hundreds of digits long. In actual programs, an algorithm called the “Extended Euclidean Algorithm” is used.

Solving $7d \equiv 1 \pmod{120}$ is the same as finding integers $d, y$ that satisfy $7d + 120y = 1$. By working backwards through the Euclidean Algorithm, this can be calculated mechanically.

$120 \div 7 = 17$ remainder $1$
Transforming this, $1 = 120 - 17 \times 7$
In other words, $-17 \times 7 \equiv 1 \pmod{120}$

In the world of modulo $120$, $-17$ has the same meaning as $120 - 17 = 103$. Therefore, $d = 103$ is found in an instant. This method can calculate very quickly no matter how huge the numbers are.

9. Another Face of RSA Encryption: Digital Signatures

The wonderful thing about RSA encryption is that it can also be used as a “digital signature” by reversing the roles of the public and private keys.

When encrypting, it was “Encrypt with public key $\Rightarrow$ Decrypt with private key”, but for digital signatures, it takes the steps “Encrypt with private key $\Rightarrow$ Decrypt with public key”.

flowchart TD A1["1. Alice creates a signature with her private key"] --> A2["S ≡ M^d (mod N)"] A2 --> A3["Sends the message M and signature S"] A3 --> B1["2. Bob verifies the signature with the public key"] B1 --> B2["Calculates M' ≡ S^e (mod N)"] B2 --> B3["Checks if M' matches M"]

Alice transforms the message using her own private key $d$ (this is the signature $S$), and sends it to Bob. Bob performs the verification calculation using Alice’s public key $e$. If the calculation result matches the original message, it simultaneously proves that “it is data that could only be created with Alice’s private key” and that “the message has not been tampered with along the way”.

10. Experiencing RSA Encryption with Programming

Exponentiation calculations that are tough by hand can be implemented very easily using Python. Below is a Python code that lets you experience the core logic of RSA encryption.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


def gcd(a, b):
 """Find the greatest common divisor"""
 while b != 0:
 a, b = b, a % b
 return a

def mod_inverse(e, phi):
 """Find the private key d (using built-in feature in Python 3.8+)"""
 return pow(e, -1, phi)

# 1. Key Generation
p, q = 11, 13
N = p * q
phi = (p - 1) * (q - 1)
e = 7
d = mod_inverse(e, phi)

print(f"Public Key: (N={N}, e={e}), Private Key: d={d}")

# 2. Encryption
message = 9
ciphertext = pow(message, e, N)
print(f"Ciphertext: {ciphertext}")

# 3. Decryption
decrypted_message = pow(ciphertext, d, N)
print(f"Decrypted Message: {decrypted_message}")

Python’s pow(base, exp, mod) function internally uses a fast algorithm called “Exponentiation by squaring”, so calculations finish in an instant even for numbers with hundreds of digits.

11. Conclusion and Future Cryptographic Technology

Based on the knowledge of high school mathematics, we have uncovered how RSA encryption works.

Difficulty of Prime Factorization: $p \times q = N$ is easy, but finding $p, q$ from $N$ is extremely difficult.
Congruence and Euler’s Theorem: With the rule $a^{\phi(N)} \equiv 1 \pmod N$, the magic trapdoor “raising to a certain power brings it back to the original” is completed.
Public and Private Keys: Anyone can encrypt, but only the legitimate receiver can decrypt.

The $N$ in currently used RSA encryption has over 600 digits, and even mobilizing all the supercomputers in the world, prime factorization would take more time than the age of the universe. However, when the “quantum computers” currently being researched become practical in the future, there is a possibility that this prime factorization will be solved in an instant by “Shor’s algorithm”. For this reason, the development of “Post-quantum cryptography” that cannot be cracked even by quantum computers is progressing rapidly worldwide.

Advanced mathematics, which is often thought of as “useless,” is actually protecting our daily lives from the ground up. RSA encryption is the best teaching material to show us the depth and beauty of such mathematics. I hope this article has helped you feel the fascination of cryptography and mathematics, even just a little.

What is Fully Homomorphic Encryption (FHE)? Explaining the Keystone of Next-Generation Security

Fri, 11 Sep 2026 11:00:00 +0900

As cloud computing and AI technologies become established as societal infrastructure, the tradeoff between “data privacy” and “data utilization” has become one of the most critical challenges. While there is a growing demand to have AI analyze highly sensitive data—such as medical records, financial information, and personal biometric data—on the cloud, many companies hesitate to send data externally due to security concerns.

Traditional encryption technologies (like AES and RSA) excel at protecting data stored in storage (Data at Rest) and data flowing over networks (Data in Transit). However, when performing processing (computations) such as searching or machine learning on the server side (Data in Use), the ciphertext must first be decrypted back into plaintext. If the server is hacked at this decrypted moment, or if a malicious internal administrator peeks at the data, it directly leads to information leakage.

The dream technology that overcomes this fundamental weakness of “decryption during processing” is Fully Homomorphic Encryption (FHE). By using FHE, it becomes possible to perform computational processing while keeping the data encrypted, without ever decrypting it, and returning only the resulting ciphertext to the client.

In this article, we will thoroughly and deeply explain FHE, the keystone of next-generation security, covering everything from its concept and history, the groundbreaking breakthrough by Craig Gentry, mathematical foundations (such as Ring-LWE), its biggest challenge “noise” and its solution (bootstrapping), up to the latest implementation libraries.

1. What is Homomorphic Encryption? Basic Concepts

“Homomorphic” is an algebraic term referring to the property where mappings can be made between sets with a certain structure while preserving the structure of the operations. “Homomorphism” in cryptography is the property where operations in the plaintext space correspond to operations in the ciphertext space.

Expressed in simple formulas, let $m_1$ and $m_2$ be plaintexts, $E(\cdot)$ be the encryption function, and $D(\cdot)$ be the decryption function. If we let $\circ$ be an operation on the plaintext (such as addition or multiplication) and $\diamond$ be an operation on the ciphertext, the following relationship holds:

$$ D(E(m_1) \diamond E(m_2)) = m_1 \circ m_2 $$

In other words, if you decrypt the result of applying some operation $\diamond$ to the ciphertexts $E(m_1)$ and $E(m_2)$, it matches the result of operating $\circ$ on the original plaintexts.

Data Flow in Cloud Computing

The architecture of cloud processing using FHE is completely different from traditional ones. The following diagram shows the flow of secure data processing utilizing FHE.

The server receives the encrypted data $E(x)$, but since it does not have the secret key, it can never know the contents of the data. However, by utilizing the properties of FHE, it can apply a function $f$ (for example, an inference model for machine learning) to the ciphertext and generate $E(f(x))$. The client receives this and decrypts it with their own secret key to obtain the desired result $y = f(x)$.

2. History of Homomorphic Encryption Evolution: PHE, SHE, FHE

Homomorphic encryption did not reach its current “fully” form all at once. It is broadly classified into three stages depending on the types and number of operations it can achieve.

Partially Homomorphic Encryption (PHE)

PHE is an encryption scheme that can perform only one of either addition or multiplication indefinitely. In fact, ciphers with this property have existed for a long time.

RSA Encryption (Homomorphism for multiplication) RSA encryption unintentionally possessed a multiplicative homomorphic property. Given plaintexts $m_1, m_2$ and a public key $(e, N)$: $$ E(m_1) = m_1^e \pmod N $$ $$ E(m_2) = m_2^e \pmod N $$ Multiplying these gives: $$ E(m_1) \times E(m_2) = (m_1 \cdot m_2)^e \pmod N = E(m_1 \times m_2) $$ Thus, the multiplication of ciphertexts corresponds to the multiplication of plaintexts.
Paillier Encryption (Homomorphism for addition) The Paillier cryptosystem, invented in 1999, has an additive homomorphic property. It has been put to practical use in applications like electronic voting (aggregating encrypted votes and decrypting only the final result).

Somewhat Homomorphic Encryption (SHE)

This scheme can execute both addition and multiplication, but there is a limit to the number of operations (circuit depth) that can be performed. Due to the accumulation of “noise,” which will be discussed later, decryption becomes impossible after a certain number of multiplications. The BGN (Boneh-Goh-Nissim) cryptosystem of 2005 falls under this category, but it had limitations in performing practical, complex computations (like deep learning).

Fully Homomorphic Encryption (FHE)

This is an encryption scheme that can execute both addition and multiplication an unlimited number of times. Similar to Turing completeness in information theory, if addition (equivalent to XOR) and multiplication (equivalent to AND) can be combined infinitely, it means that in principle, any computable function or algorithm can be executed while remaining encrypted.

FHE was long called the “holy grail of cryptography” and was even said to be impossible to realize. However, in 2009, Craig Gentry, who was a doctoral student at Stanford University at the time, proposed the first FHE scheme using Ideal Lattices, sending shockwaves through the world.

3. Mathematical Foundations of FHE: The LWE Problem and Ring-LWE

Many of the current mainstream FHE schemes are based on the LWE (Learning With Errors) problem, a mathematical hard problem in “Lattice-based Cryptography,” which is also known as Post-Quantum Cryptography.

Intuitive Understanding of the LWE Problem

Solving a system of linear equations is easy if you use methods like Gaussian elimination.

$$ \begin{cases} 3s_1 + 4s_2 + 2s_3 \equiv 12 \pmod{17} \\ 1s_1 + 9s_2 + 5s_3 \equiv 8 \pmod{17} \\ \vdots \end{cases} $$

However, what happens if we add a very small “random error (noise)” $e$ to the results of these equations?

$$ \begin{cases} 3s_1 + 4s_2 + 2s_3 + e_1 \equiv 13 \pmod{17} \\ 1s_1 + 9s_2 + 5s_3 + e_2 \equiv 7 \pmod{17} \\ \vdots \end{cases} $$

With just the addition of this error $e$, the problem of finding the secret variable vector $\vec{s}$ transforms into an NP-hard problem that is difficult to decipher even using current supercomputers or quantum computers. This is the LWE problem.

Ring-LWE Problem (RLWE)

The standard LWE problem involves matrix operations, which means the key size is extremely large (sometimes in gigabytes) and computational efficiency is poor. To solve this, the Ring-LWE (RLWE) problem, which uses operations over polynomial rings, was introduced.

In RLWE, elements belong to the polynomial ring $R_q = \mathbb{Z}_q[x] / (x^N + 1)$ (where $N$ is a power of 2, and $q$ is the modulus prime). Let the secret key be a polynomial $s(x)$, and with a random polynomial $a(x)$ and a small noise polynomial $e(x)$, the public key becomes the following pair:

$$ (a(x), b(x)) \quad \text{where} \quad b(x) = -a(x) \cdot s(x) + e(x) \pmod q $$

During encryption, the plaintext $m(x)$ is encoded using the properties of this polynomial to generate the ciphertext.

4. The Biggest Barrier “Noise” and Gentry’s Bootstrapping

The most important concept in understanding FHE is “noise management.”

In LWE/RLWE-based cryptography, small “noise (errors)” are intentionally included to ensure security. The process of decrypting a ciphertext $c$ of a plaintext $m$ can be roughly represented by the following formula:

$$ D(c) = (c \cdot s) \pmod q = m + \text{noise} $$

During decryption, this noise is removed through rounding processes or similar to obtain the correct plaintext $m$. However, when homomorphic operations (especially multiplication) are performed between ciphertexts, this noise is dramatically amplified.

Homomorphic Addition: Noise increases additively ($e_1 + e_2$). This is a relatively gradual increase.
Mathematical Representation of Homomorphism by Homomorphic Addition: $$ E(m_1) \oplus E(m_2) = E(m_1 + m_2) $$
Homomorphic Multiplication: Noise explodes multiplicatively (because it includes terms like $e_1 \times e_2$). After just a few multiplications, the noise exceeds the threshold $q/2$, preventing correct rounding and causing decryption to fail.
Mathematical Representation of Homomorphism by Homomorphic Multiplication: $$ E(m_1) \otimes E(m_2) = E(m_1 \times m_2) $$

This is the reason why FHE could not be realized for a long time and remained at the level of SHE (with a limited number of operations).

The Magic of Bootstrapping

Craig Gentry’s genius contribution was inventing a noise reduction technique called “bootstrapping.” This was a paradigm shift in cryptography.

Intuitively, it is the operation of “‘decrypting’ the ciphertext to clean it while it remains encrypted, and putting it into a new ciphertext before it becomes too noisy and breaks.”

Suppose we have a highly noisy ciphertext $C_{noisy}$.
The client provides the server in advance with the secret key $sk$ “encrypted with the public key,” $E_{pk}(sk)$ (this is called the bootstrapping key).
The server runs a Decryption Circuit homomorphically on $C_{noisy}$.
Specifically, it performs a “decryption within the encrypted space” on $E_{pk}(C_{noisy})$ using $E_{pk}(sk)$.
Since this decryption circuit itself is a homomorphic operation, it generates new noise, but the noise of the newly output ciphertext $C_{fresh}$ is reset to a fixed “constant level.”

graph LR A["High noise ciphertext C_noisy"] --> B["Homomorphic decryption circuit (Eval_Dec)"] C["Encrypted secret key E(sk)"] --> B B --> D["Low noise ciphertext C_fresh"] style B fill:#ffeeba,stroke:#ffc107

By executing this bootstrapping periodically during computation, it theoretically became possible to compute circuits of infinite depth (achieving FHE). However, Gentry’s early scheme was desperately expensive computationally, with a single bootstrapping operation taking anywhere from tens of minutes to hours.

5. Generations of FHE and the Evolution of Major Schemes

In the race toward practical FHE, cryptographers around the world have competed to improve the algorithms. Currently, FHE is mainly classified into four generations or families.

2nd Generation: Exact Integer Arithmetic (BGV, BFV)

The BGV (Brakerski-Gentry-Vaikuntanathan) and BFV (Brakerski/Fan-Vercauteren) schemes appeared between 2011 and 2012. These are based on RLWE and are suitable for integer modular arithmetic (exact calculations). They support batching techniques like SIMD (Single Instruction, Multiple Data), characterized by the ability to pack thousands of data slots into a single large polynomial ciphertext and compute them in parallel all at once.

3rd Generation: Accelerated Bootstrapping (GSW, FHEW, TFHE)

The GSW (Gentry-Sahai-Waters) scheme of 2013 made the structure of FHE simpler. This was developed further into TFHE (Fast Fully Homomorphic Encryption over the Torus), one of the mainstream schemes today. The hallmark of TFHE is its extremely fast bootstrapping (on the order of milliseconds). It excels at gate-level operations (logic circuits like AND, XOR), and since the ciphertext size is relatively small, it is suited for fast evaluation of arbitrary logic circuits.

4th Generation: Specialization for Approximate Calculation and Machine Learning (CKKS)

The CKKS (Cheon-Kim-Kim-Song) scheme proposed by Cheon et al. in 2017 can be called the definitive technology for privacy protection in current AI and machine learning. While previous FHEs insisted on “exact integer calculations,” CKKS supports “approximate calculations of floating-point numbers” while remaining encrypted. It demonstrates overwhelming performance in real number calculations where small errors are tolerable, such as the training and inference of neural networks.

The table below summarizes how to choose a scheme by purpose.

Scheme Name	Preferred Data Type	Recommended Use Cases	Features
BFV / BGV	Integer	Exact statistical calculations, financial data aggregation, DB queries	High throughput via SIMD batching
CKKS	Real/Complex	Machine learning (DNN, logistic regression), signal processing	Acceleration via approximate calculation, rescaling
TFHE	Boolean	Arbitrary logic circuits, string search, evaluation of non-linear functions	Ultra-fast bootstrapping (millisecond range)

6. Practice: FHE Libraries and Conceptual Code

Today, many open-source libraries are provided that allow you to use FHE without deep cryptographic knowledge.

Microsoft SEAL (Simple Encrypted Arithmetic Library): A C++ library supporting BFV, BGV, and CKKS. One of the industry standards. Its Python binding, TenSEAL, is popular among AI engineers.
Zama (Concrete): A framework based on TFHE. You can write in Rust/Python, and it provides functionality (Concrete ML) to compile existing PyTorch models and run them on FHE.
OpenFHE: The successor to PALISADE, a comprehensive C++ library supporting all major schemes.

Example of FHE Programming using Python (TenSEAL)

Here, we show a conceptual Python code example using the CKKS scheme to add and multiply real number vectors while they remain encrypted.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


import tenseal as ts

# 1. Context setup (including key generation)
# Use CKKS scheme, set polynomial degree to 8192
context = ts.context(
 ts.SCHEME_TYPE.CKKS,
 poly_modulus_degree=8192,
 coeff_mod_bit_sizes=[60, 40, 40, 60]
)
context.generate_galois_keys()
context.global_scale = 2**40 # Scaling factor for real numbers

# 2. Client side: Data encryption
vector1 = [1.5, 2.5, 3.5]
vector2 = [2.0, 3.0, 4.0]

# Convert plaintext vectors to ciphertexts (should be executed on the client side)
enc_v1 = ts.ckks_vector(context, vector1)
enc_v2 = ts.ckks_vector(context, vector2)

# 3. Server side: Computations while encrypted (Protection of Data in Use)
# The server does not know the plaintexts but can perform addition and multiplication
enc_add = enc_v1 + enc_v2
enc_mul = enc_v1 * enc_v2

# 4. Client side: Decryption of results
# Only the client with the secret key can view the results
res_add = enc_add.decrypt()
res_mul = enc_mul.decrypt()

print(f"Decrypted addition result: {res_add}")
# Example output: [3.5000001, 5.5000001, 7.5000002] (Includes minute errors due to approximate calculation)

print(f"Decrypted multiplication result: {res_mul}")
# Example output: [3.0000002, 7.5000005, 14.0000003]

As you can see from the code above, you can intuitively describe computations between ciphertexts by overloading normal Python operators, such as enc_v1 + enc_v2. On the server side, vector operations are completed without knowing the contents of the vectors.

7. FHE Challenges: Performance and Hardware Acceleration

While FHE provides theoretically perfect security, its biggest challenge for practical use is “performance overhead.”

Computational Overhead: Compared to computing in plaintext, computing in ciphertext is thousands to tens of thousands of times slower on a CPU. Polynomial multiplications and bootstrapping require massive amounts of FFT (Fast Fourier Transform) or NTT (Number Theoretic Transform) calculations.
Ciphertext Expansion: A few bytes of plaintext can expand to several megabytes when encrypted. This puts severe pressure on memory bandwidth and network bandwidth.

Approaches to Hardware Solutions

To overcome this overhead, the development of dedicated FHE hardware accelerators (ASIC, FPGA, GPU support) is progressing worldwide.

GPU Acceleration: Efforts are underway to parallelize NTT operations and bootstrapping using powerful GPUs from NVIDIA and others, with reports of speeds tens of times faster than software implementations (e.g., 100x.ai, Zama’s TFHE-rs CUDA backend).
DARPA DPRIVE Project: The US Defense Advanced Research Projects Agency (DARPA) is promoting the “DPRIVE (Data Protection in Virtual Environments)” project to develop dedicated hardware to bring the computational speed of FHE to parity with plaintext processing (within a 10x overhead). Intel, Microsoft, and Intellectual Ventures are participating.
Emergence of FPUs (FHE Processing Units): Startups like Cornami and Optalysys are embarking on the development of FHE-specific chips using optical computing and specialized silicon architectures.

In the near future, an era may come where “FPUs” become a standard feature in server and cloud infrastructure, just like NPUs (Neural Processing Units) in AI.

8. Expected Use Cases

Now that FHE is approaching practical speeds, disruptive innovations are expected in areas such as:

Privacy Protection in Medical and Genomic Analysis: By having a cloud AI learn from patients’ medical records and DNA data held by multiple hospitals while keeping it encrypted with FHE, highly accurate cancer diagnostic models and new drug development can be performed without violating privacy laws (like HIPAA or GDPR).
Fraud Detection and Anti-Money Laundering (AML) for Financial Institutions: Competing banks can cross-analyze data in an encrypted state to detect massive illegal money transfer networks, without revealing customer account information or transaction histories to each other.
Secure AI Inference APIs (MaaS: Model as a Service): Users encrypt their voice, facial images, and prompts before sending them to AI services (like LLMs such as ChatGPT). The AI provider generates the answer without ever knowing the user’s input and returns it as a ciphertext. This completely dispels the concern of “AI learning or peeking at personal information.”

9. Conclusion: The Future of Cryptography is “Unseen Computation”

Just as the invention of public key cryptography (RSA) in the 1970s enabled secure communication on the Internet (such as HTTPS), Craig Gentry’s invention of FHE is one of the most important milestones in the history of cryptography.

Today, Fully Homomorphic Encryption (FHE) has leapt from the theories of laboratories into the stage where Microsoft, IBM, Intel, Google, and many startups are fiercely competing toward practical application. While challenges regarding computational cost and data size still exist, thanks to the refinement of algorithms and the evolution of hardware accelerators, performance improvements continue at a pace exceeding Moore’s Law.

In a few years, “computing data while keeping it encrypted” will not be something special, but will likely become the standard data protection best practice in cloud services. FHE is the keystone of next-generation security, realizing the ultimate combination of privacy and data utilization in a data-driven society.

Mathematical Foundations of Elliptic Curve Cryptography (ECC) and Implementation in C++

Fri, 11 Sep 2026 10:00:00 +0900

Mathematical Foundations of Elliptic Curve Cryptography (ECC) and Implementation in C++

In modern cryptographic technology, Elliptic Curve Cryptography (ECC) plays a tremendously important role. It is no exaggeration to say that the foundation of trust in our modern digital society—from everyday Internet communications (HTTPS/TLS) and smartphone secure enclaves, to server authentication via SSH, passwordless authentication like FIDO, and even crypto assets like Bitcoin and Ethereum—is supported by ECC.

In this article, we will thoroughly explain how elliptic curve cryptography works, starting from the beautiful yet complex mathematical theory behind it (algebraic geometry over finite fields), to actual implementation methods in C++, and even secure coding techniques to prevent side-channel attacks (timing attacks), with an overwhelming volume of detail.

1. Why Elliptic Curve Cryptography? (Comparison with RSA)

For a long time, RSA cryptography was synonymous with public-key cryptography. The security of RSA relies on the “difficulty of factoring large composite numbers.” However, as the computational power of computers has increased, it has become necessary to continuously lengthen the RSA key size (number of modulus bits) to maintain security. Currently, a key size of at least 2048 bits is recommended, and 3072 or 4096 bits if greater security is desired.

In contrast, Elliptic Curve Cryptography (ECC) bases its security on a different mathematical difficulty: the “Elliptic Curve Discrete Logarithm Problem (ECDLP)”. To this day, no efficient algorithms (such as sub-exponential time algorithms) for solving the ECDLP have been discovered, and even the most efficient known attack methods require exponential time.

Because of this property, ECC has the decisive advantage of being able to achieve security strength equivalent to RSA with significantly shorter key lengths.

Security Strength (bits)	RSA Key Length (bits)	ECC Key Length (bits)	Key Length Ratio
80	1024	160	1:6
112	2048	224	1:9
128	3072	256	1:12
192	7680	384	1:20
256	15360	512	1:30

As shown in the table above, to achieve a security strength of 128 bits (currently a standard strength), RSA requires a 3072-bit key, whereas ECC requires only 256 bits. This reduces computational complexity, lowers memory usage, and saves network bandwidth, making ECC overwhelmingly superior, especially in resource-constrained IoT devices and smart card environments.

2. Mathematical Preparation: The World of Group Theory and Finite Fields

To truly understand elliptic curve cryptography, it is necessary to grasp the basic concepts of abstract algebra (group theory and field theory). Here, we briefly summarize the prerequisite knowledge for constructing ECC.

2.1. Groups and Abelian Groups

A Group is a pair consisting of a set $G$ and a binary operation on that set (here we’ll use addition $+$), denoted as $(G, +)$, which satisfies the following four axioms:

Closure: For any $a, b \in G$, $a + b \in G$.
Associativity: For any $a, b, c \in G$, $(a + b) + c = a + (b + c)$.
Identity element: There exists an element $e \in G$ such that for any $a \in G$, $a + e = e + a = a$. For additive groups, this identity element is usually denoted as $0$ or $\mathcal{O}$.
Inverse element: For any $a \in G$, there exists an element $b \in G$ such that $a + b = b + a = e$. This $b$ is denoted as $-a$.

Furthermore, a group that satisfies the following condition, where the result does not change even if the order of the operation is swapped, is called an Abelian group (commutative group).

Commutativity: For any $a, b \in G$, $a + b = b + a$.

The set of points on an elliptic curve, by defining a specific addition rule, constitutes this Abelian group.

2.2. Finite Fields

In cryptography, we do not use fields with continuous and infinite elements like real or complex numbers, but rather Finite Fields (or Galois Fields), which have a finite number of elements.

The most basic finite field is the prime field $\mathbb{F}_p$ using a prime number $p$. This defines the four basic arithmetic operations (addition, subtraction, multiplication, division) modulo $p$ (the remainder when divided by $p$) on the set of integers $\{0, 1, 2, \dots, p-1\}$.

Addition: $(a + b) \pmod p$
Subtraction: $(a - b) \pmod p$
Multiplication: $(a \times b) \pmod p$
Division: $a \times b^{-1} \pmod p$ (where $b^{-1}$ is the modular multiplicative inverse of $b$ modulo $p$)

Calculating the Modular Multiplicative Inverse is extremely important in cryptographic implementations. To find $b^{-1}$ satisfying $b \times b^{-1} \equiv 1 \pmod p$, the following two main algorithms are used:

Extended Euclidean Algorithm: Fast, but depending on the implementation, the processing time can depend on the input values, leading to a risk of timing attacks.
Fermat’s Little Theorem: When $p$ is prime and $b \neq 0$, $b^{p-1} \equiv 1 \pmod p$ holds. Dividing both sides by $b$ yields $b^{p-2} \equiv b^{-1} \pmod p$. That is, the inverse is found by raising $b$ to the power of $p-2$. Exponentiation is easier to implement in constant time, so this method is preferred in cryptographic implementations.

3. Elliptic Curve Equations and Geometry

3.1. Weierstrass Normal Form

An Elliptic Curve is generally a plane curve defined by the following equation, known as the Weierstrass normal form.

$$ y^2 = x^3 + ax + b $$

Here, $a$ and $b$ are constants, and as a condition for the curve to have no singular points (self-intersections or cusps) (i.e., to be a smooth curve), the following Discriminant $\Delta$ must not be zero.

$$ \Delta = -16(4a^3 + 27b^2) \neq 0 $$

Since curves with singular points compromise cryptographic security, coefficients $a, b$ that satisfy this condition are always chosen.

3.2. Point at Infinity

To make an elliptic curve a mathematically complete group, a virtual point called the “Point at Infinity” is introduced in addition to the points on the plane. This is denoted as $\mathcal{O}$.

The point at infinity $\mathcal{O}$ is defined as the point where all vertical lines intersect infinitely far away. In group theory, this point at infinity $\mathcal{O}$ functions as the identity element (zero) for addition.

In other words, for any point $P$ on the curve, the following holds:

$$ P + \mathcal{O} = \mathcal{O} + P = P $$

Also, the inverse $-P$ of point $P = (x, y)$ is defined as the point symmetrical with respect to the x-axis, $(x, -y)$. Therefore:

$$ P + (-P) = \mathcal{O} $$

holds.

4. Group Operations on Elliptic Curves (Point Addition and Point Doubling)

The core of elliptic curve cryptography is the operation called “Addition” between points on the curve. This is different from the addition of ordinary integers and is defined based on geometric operations.

4.1. Geometric Addition (Tangent and Chord Method)

The procedure for finding a new point $R$ ($R = P + Q$) by adding two distinct points $P$ and $Q$ on the curve is as follows:

Draw a straight line (chord) connecting point $P$ and point $Q$.
This straight line will definitely intersect the elliptic curve at a third point (let’s call it $-R$). (*Based on a theorem in algebraic geometry)
The point $R$ we are looking for is the point obtained by reflecting the intersection point $-R$ symmetrically across the x-axis (the point with the inverted sign of the y-coordinate).

graph TD Step1["Draw a line connecting P(x1, y1) and Q(x2, y2)"] --> Step2["Find the 3rd intersection -R with the curve"] Step2 --> Step3["Reflect -R across the x-axis to get R(x3, y3)"] Step3 -.-> Result["This is R = P + Q"]

4.2. Point Doubling

When adding the same point $P$ to point $P$ ($P + P = 2P$), we cannot draw a line connecting two points. In this case, we draw the tangent line to the curve at point $P$.

Draw the tangent line to the curve at point $P$.
This tangent line intersects the curve at another point $-R$.
The point $R = 2P$ we are looking for is obtained by reflecting the intersection symmetrically across the x-axis.

4.3. Algebraic Calculation Formulas

We translate the geometric operations into algebraic formulas so they can be computed by a computer. All operations are performed over the finite field $\mathbb{F}_p$ (modulo $p$).

Let point $P = (x_1, y_1)$ and point $Q = (x_2, y_2)$. Also, let the resulting point be $R = P + Q = (x_3, y_3)$.

Let the slope of the line be $\lambda$ (lambda).

[Case 1: When $P \neq Q$ (Point Addition)] The slope $\lambda$ is the rate of change between the two points.

$$ \lambda \equiv \frac{y_2 - y_1}{x_2 - x_1} \pmod p $$ $$ \lambda \equiv (y_2 - y_1) \cdot (x_2 - x_1)^{-1} \pmod p $$

Using this $\lambda$, $x_3, y_3$ are found as follows:

$$ x_3 \equiv \lambda^2 - x_1 - x_2 \pmod p $$ $$ y_3 \equiv \lambda(x_1 - x_3) - y_1 \pmod p $$

[Case 2: When $P = Q$ (Point Doubling)] The slope $\lambda$ becomes the slope of the tangent line obtained by differentiation. (We implicitly differentiate $y^2 = x^3 + ax + b$)

$$ 2y \cdot y' = 3x^2 + a \implies y' = \frac{3x^2 + a}{2y} $$

Therefore,

$$ \lambda \equiv (3x_1^2 + a) \cdot (2y_1)^{-1} \pmod p $$

The formulas for $x_3, y_3$ take the same form as addition, but since $x_2 = x_1$, they are as follows:

$$ x_3 \equiv \lambda^2 - 2x_1 \pmod p $$ $$ y_3 \equiv \lambda(x_1 - x_3) - y_1 \pmod p $$

[!IMPORTANT] These formulas include division (calculation of modular inverses), such as $(x_2 - x_1)^{-1}$ and $(2y_1)^{-1}$. Since calculating modular inverses incurs a very high computational cost, practical implementations generally use projective coordinate systems like “Jacobian Coordinates”, which delay division.

5. Scalar Multiplication and the Elliptic Curve Discrete Logarithm Problem (ECDLP)

In elliptic curve cryptography, the operation that requires the most computation and forms the core of its security is Scalar Multiplication.

5.1. What is Scalar Multiplication?

The operation of adding a point $P$ to itself $k$ times is called scalar multiplication, denoted as $kP$.

$$ kP = \underbrace{P + P + \dots + P}_{k \text{ times}} $$

Here, $k$ is a very large integer (for example, a 256-bit integer).

5.2. Elliptic Curve Discrete Logarithm Problem (ECDLP)

The security of elliptic curve cryptography depends on the difficulty of the following problem.

Elliptic Curve Discrete Logarithm Problem (ECDLP) Given a known point $P$ (base point) and the resulting point $Q$, find the scalar $k$ that satisfies $Q = kP$.

Calculating $Q$ from $k$ and $P$ (forward direction) is easy (polynomial time) using the algorithm described below, but inversely calculating $k$ from $P$ and $Q$ (reverse direction) is practically impossible as there is no efficient solution other than exhaustive search (a one-way function). In cryptographic protocols, $k$ corresponds to the “private key” and $Q$ to the “public key”.

5.3. Double-and-Add Algorithm

When $k$ is a huge number (e.g., $2^{256}$), naively adding $P$ for $k$ times will not finish even if the life of the universe ends. Therefore, to perform scalar multiplication quickly, the Double-and-Add method (binary method) is used.

This is the elliptic curve version of the “exponentiation by squaring” method used for rapid calculation of integer powers. The scalar $k$ is represented in binary, and processed sequentially starting from the most significant bit.

Initialize the point $R$ holding the result to $\mathcal{O}$.
Repeat the following from the most significant bit to the least significant bit of $k$:
- Double $R$ (Point Doubling: $R = 2R$)
- If the current bit is 1, add $P$ to $R$ (Point Addition: $R = R + P$)

With this algorithm, the computational complexity is dramatically reduced from $O(k)$ to $O(\log_2 k)$, enabling calculation in a realistic amount of time (milliseconds).

6. Elliptic Curve Diffie-Hellman (ECDH) Key Exchange

Here, we explain the mechanics of the ECDH (Elliptic Curve Diffie-Hellman) key exchange protocol, which is the most representative application of ECC. ECDH is a mechanism for Alice and Bob to securely generate and share a common secret key (session key) over a communication channel that could be wiretapped (it is the core of the TLS handshake).

[Prerequisite Parameters (Domain Parameters)] Both parties share the elliptic curve $E$, a prime number $p$, and a base point $G$ to be used beforehand. (e.g., NIST P-256 or secp256k1)

sequenceDiagram participant Alice as "Alice" participant Bob as "Bob" Note over Alice,Bob: "Public parameters: Curve E, Base point G, Prime p" Alice->>Alice: "Generate private key d_A (random integer)" Alice->>Alice: "Calculate public key Q_A = d_A * G" Bob->>Bob: "Generate private key d_B (random integer)" Bob->>Bob: "Calculate public key Q_B = d_B * G" Alice->>Bob: "Send public key Q_A (plaintext)" Bob->>Alice: "Send public key Q_B (plaintext)" Alice->>Alice: "Calculate shared secret S = d_A * Q_B" Bob->>Bob: "Calculate shared secret S = d_B * Q_A" Note over Alice,Bob: "S = d_A * (d_B * G) = d_B * (d_A * G) = (d_A * d_B) * G" Note over Alice,Bob: "Both parties' calculation results S perfectly match!"

An eavesdropper (Eve) can intercept $G$, $Q_A$, and $Q_B$ flowing over the communication path, but due to the difficulty of the ECDLP, she cannot determine Alice’s private key $d_A$ from $Q_A = d_A \cdot G$. Also, even if she multiplies $Q_A$ and $Q_B$, it will not result in the shared key $S$, so the eavesdropper cannot calculate $S$.

7. Implementation Pitfalls: Side-Channel Attacks and Countermeasures

Even a theoretically perfect cryptographic algorithm can harbor vulnerabilities introduced during the process of implementing it as a program. This is known as a “Side-Channel Attack”.

7.1. Timing Attack

Let’s look back at the Double-and-Add algorithm mentioned earlier.

1
2
3
4
5
6
7
8


// Vulnerable pseudo-code for Double-and-Add
Point R = Point::Infinity;
for (int i = 255; i >= 0; i--) {
 R = PointDoubling(R); // Always executed
 if (bit(k, i) == 1) {
 R = PointAddition(R, P); // Executed ONLY when bit is 1!
 }
}

This implementation has a fatal flaw. Because Point Addition is executed when the bit is 1, the calculation time is slightly longer than when the bit is 0. Also, the behavior of the processor’s branch prediction and cache memory will change. By statistically observing this minute difference in calculation time (or power consumption) thousands of times, an attacker can completely recover the bit sequence of the private key $k$, one bit at a time. This is a timing attack.

7.2. Constant-Time Implementation: Montgomery Ladder

To prevent timing attacks, it is necessary to adopt algorithms where the sequence of executed instructions and the calculation time are always constant (Constant-Time), regardless of the bit values of the private key.

A representative example of this is the Montgomery Ladder.

graph TD Start["Initialization: R0 = O, R1 = P"] --> LoopStart["For each bit i (from MSB)"] LoopStart --> Cond{"Value of k_i?"} Cond -->|0| Branch0["R1 = R0 + R1
R0 = 2 * R0"] Cond -->|1| Branch1["R0 = R0 + R1
R1 = 2 * R1"] Branch0 --> LoopEnd["Next bit"] Branch1 --> LoopEnd LoopEnd --> LoopStart LoopStart -.->|"All bits processed"| End["End: R0 is the result (kP)"]

The beauty of the Montgomery Ladder is that whether the bit is 0 or 1, “exactly one Point Addition and one Point Doubling” are always executed. This completely eliminates the data dependency of the computation time.

However, if a branch (if (k_i == 0)) itself exists, the risk of execution time fluctuating due to compiler optimization and CPU branch prediction remains. Therefore, in actual Constant-Time implementations, conditional branches (if statements) are eliminated, and a Conditional Swap using bitwise operations is utilized.

8. Implementation of Elliptic Curve Cryptography in C++

From here, we will translate the theory into C++ code. While practical cryptographic libraries (like OpenSSL or libsodium) use highly advanced assembly optimizations and Jacobian coordinates, we present the skeleton of an easy-to-understand Constant-Time implementation using affine coordinates to deepen mathematical understanding.

We assume the use of boost::multiprecision::cpp_int for operations on huge integers.

8.1. Modular Arithmetic and Inverses

First, we define helper functions for operations over finite fields. We implement inverse calculation using Fermat’s Little Theorem.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


#include <iostream>
#include <vector>
#include <stdexcept>
#include <boost/multiprecision/cpp_int.hpp>

using namespace boost::multiprecision;

// Prime p and parameters for secp256k1 as an example
const cpp_int p("0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F");
const cpp_int a = 0;
const cpp_int b = 7;

// Modulo operation returning a positive remainder
cpp_int mod(cpp_int x, cpp_int m) {
 cpp_int r = x % m;
 return r < 0 ? r + m : r;
}

// Modular exponentiation (x^y mod m)
cpp_int powerMod(cpp_int base, cpp_int exp, cpp_int m) {
 cpp_int res = 1;
 base = mod(base, m);
 while (exp > 0) {
 if (exp % 2 == 1) res = mod(res * base, m);
 base = mod(base * base, m);
 exp /= 2;
 }
 return res;
}

// Modular inverse using Fermat's Little Theorem
cpp_int modInverse(cpp_int n, cpp_int m) {
 // Assuming m is prime: n^(m-2) ≡ n^(-1) mod m
 return powerMod(n, m - 2, m);
}

8.2. Point Representation and Group Operations (Addition/Doubling)

We implement the Point structure, which manages the point at infinity with a flag, and the addition formulas.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41


struct Point {
 cpp_int x;
 cpp_int y;
 bool isInfinity;

 // Creation of the point at infinity
 Point() : x(0), y(0), isInfinity(true) {}

 // Creation of a normal point
 Point(cpp_int x, cpp_int y) : x(x), y(y), isInfinity(false) {}
};

// Addition of points on the elliptic curve (R = P + Q)
Point pointAdd(const Point& P, const Point& Q) {
 if (P.isInfinity) return Q;
 if (Q.isInfinity) return P;

 if (P.x == Q.x && mod(P.y + Q.y, p) == 0) {
 return Point(); // P + (-P) = Point at infinity
 }

 cpp_int lambda;
 if (P.x == Q.x && P.y == Q.y) {
 // Point Doubling (when P = Q)
 // lambda = (3x^2 + a) / 2y
 cpp_int num = mod(3 * P.x * P.x + a, p);
 cpp_int den = modInverse(mod(2 * P.y, p), p);
 lambda = mod(num * den, p);
 } else {
 // Point Addition (when P != Q)
 // lambda = (y2 - y1) / (x2 - x1)
 cpp_int num = mod(Q.y - P.y, p);
 cpp_int den = modInverse(mod(Q.x - P.x, p), p);
 lambda = mod(num * den, p);
 }

 cpp_int x3 = mod(lambda * lambda - P.x - Q.x, p);
 cpp_int y3 = mod(lambda * (P.x - x3) - P.y, p);

 return Point(x3, y3);
}

8.3. Implementation of Constant-Time Conditional Swap

When swapping the contents of variables based on the bit value of the private key, we use only bitwise operations (masks) without using an if statement. This ensures the execution path is completely constant.

[!TIP] In an actual implementation, dynamically allocated multiple-precision integer classes like cpp_int are not suitable for Constant-Time processing. This is because timing information leaks due to variations in memory allocation and array sizes. Practical libraries represent them with fixed lengths (for example, an array of 4 uint64_t elements) and implement bit-level masking. The following is a conceptual example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


// Conceptual Constant-Time Swap (assuming fixed-length integers)
// If bit is 1, swap P1 and P2; if 0, do not swap
void cswap(Point& P1, Point& P2, uint8_t bit) {
 // bit is 0 or 1. Mask is all 1s (0xFF..) if bit=1, all 0s if 0.
 // (Here we assume each word of a fixed-length BigInt class is w for explanation)
 /*
 uint64_t mask = 0 - (uint64_t)bit;
 for (int i = 0; i < NUM_WORDS; i++) {
 uint64_t dummy = mask & (P1.x.words[i] ^ P2.x.words[i]);
 P1.x.words[i] ^= dummy;
 P2.x.words[i] ^= dummy;
 // Process y-coordinate and isInfinity flag similarly
 }
 */

 // * Perfect constant-time swapping is difficult with boost::multiprecision,
 // so here we limit it to simulating with a branch for the sake of understanding the logic.
 if (bit == 1) {
 std::swap(P1, P2);
 }
}

8.4. Scalar Multiplication using the Montgomery Ladder

We combine the aforementioned pointAdd and cswap to implement secure scalar multiplication.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


// Scalar multiplication k * P (Montgomery Ladder method)
Point scalarMultiply(const Point& P, cpp_int k) {
 Point R0 = Point(); // Point at infinity
 Point R1 = P;

 // Get the bit length of k (256 bits for secp256k1)
 int numBits = 256;

 for (int i = numBits - 1; i >= 0; i--) {
 // Get the value of the i-th bit (0 or 1)
 uint8_t bit = static_cast<uint8_t>(bit_test(k, i) ? 1 : 0);

 // Swap R0 and R1 if bit == 1
 cswap(R0, R1, bit);

 // Always execute the same operations (Point Addition and Point Doubling)
 R1 = pointAdd(R0, R1);
 R0 = pointAdd(R0, R0);

 // Swap back to restore state if bit == 1
 cswap(R0, R1, bit);
 }

 return R0;
}

With this implementation logic, whether each bit of the scalar $k$ is 0 or 1, the operations executed within each loop iteration (cswap $\to$ pointAdd $\to$ pointAdd $\to$ cswap) follow the exact same flow. This powerfully prevents the leakage of secret information through differences in timing or cache access patterns.

9. Conclusion

At first glance, Elliptic Curve Cryptography (ECC) might seem puzzling: “Why does a geometric operation like drawing a line and reflecting the intersection point become cryptography?” However, by mapping it to the discrete world of finite fields, an excellent one-way function (the discrete logarithm problem) can be constructed, making it a product of the miraculous fusion of mathematics and cryptography.

In this article, we covered the following key points:

Superiority over RSA: Provides strong security with very short key lengths, making it ideal for the modern mobile and IoT era.
Basics of Group Theory and Finite Fields: The mathematical structure that forms the foundation of ECC.
Addition and Doubling Formulas: Implementation methods of algebraic group operations using Weierstrass equations.
Threat of Side-Channel Attacks: Conditional branches dependent on private key bits create fatal vulnerabilities.
Constant-Time Implementation: C++ coding techniques that uniformize hardware-level behavior using the Montgomery Ladder and Conditional Swap to prevent attacks.

Writing your own cryptographic library to run in a production environment is highly discouraged (“Don’t roll your own crypto”) because the security risks are extremely high. However, deeply understanding the underlying algorithms and mathematical background should serve as an invaluable and powerful weapon for engineers designing and operating more secure and performant systems.

In the next article, we would like to dig even deeper into the mechanics of the ECDSA (Elliptic Curve Digital Signature Algorithm), which is a digital signature algorithm using these elliptic curves, as well as Schnorr signatures, which are adopted in Bitcoin.

Is There an Algorithm Beyond GNFS (General Number Field Sieve)?

Fri, 11 Sep 2026 09:00:00 +0900

1. Introduction: Integer Factorization and the Foundation of Modern Cryptography

The security of internet communication in modern society heavily relies on the security of the RSA cryptosystem, a public-key encryption. And the security of RSA is based on the mathematical assumption of “the difficulty of factoring huge composite numbers”. If an extremely efficient integer factorization algorithm were discovered, the world’s communication infrastructure would collapse from its foundation.

Currently, the General Number Field Sieve (GNFS) reigns as the fastest and strongest algorithm for factoring huge integers using classical computers. GNFS was born as an extension of the Special Number Field Sieve (SNFS) proposed in the late 1980s, and to this day, it has established factorization records for huge composite numbers such as RSA-768 and RSA-250.

However, cryptographers and mathematicians always harbor the following questions: “Is there a classical algorithm that surpasses GNFS?” “Where are the limits of classical computers?” And, “How will quantum computers break through this situation?”

In this article, we thoroughly dissect the profound mathematical structures behind GNFS and conduct a detailed technical analysis of polynomial selection, the sieving phase, and the linear algebra step using the block Wiedemann method. Furthermore, we consider extension methods of GNFS such as Coppersmith’s improvements, and compare and explain the decisive differences between classical sub-exponential time algorithms and quantum polynomial time algorithms from a mathematical perspective.

2. Asymptotic Complexity and L-notation

When evaluating the computational complexity of integer factorization algorithms, instead of standard polynomial time notation (such as $O(n^k)$), L-notation is used to express the sub-exponential time relative to the number of digits of the input $n$. L-notation is defined as follows:

$$ L_n[\alpha, c] = \exp \left( (c + o(1)) (\ln n)^\alpha (\ln \ln n)^{1-\alpha} \right) $$

Here, $n$ is the integer to be factored, and $\ln n$ is the natural logarithm, which is proportional to the bit length of $n$.

When $\alpha = 0$: $L_n[0, c] = \exp(c \ln \ln n) = (\ln n)^c$, representing polynomial time relative to the bit length.
When $\alpha = 1$: $L_n[1, c] = \exp(c \ln n) = n^c$, representing exponential time relative to the bit length.
When $0 < \alpha < 1$: It becomes sub-exponential time, positioned between polynomial time and exponential time.

The history of the evolution of past integer factorization algorithms has also been a history of gradually reducing this value of $\alpha$.

Continued Fraction Factorization (CFRAC) and Multiple Polynomial Quadratic Sieve (MPQS): Belong to the class of $\alpha = 1/2$, with a complexity of around $L_n[1/2, 1]$.
General Number Field Sieve (GNFS): Achieved $\alpha = 1/3$, boasting a complexity of $L_n[1/3, (64/9)^{1/3}]$, the fastest among currently known classical algorithms.

3. The Full Picture and Mathematical Structure of the GNFS Algorithm

GNFS has a very complex and advanced mathematical foundation. The basic idea is an extension of Fermat’s Little Theorem and the Quadratic Sieve (QS), finding a non-trivial pair $(X, Y)$ that satisfies the congruence $X^2 \equiv Y^2 \pmod n$ and $X \not\equiv \pm Y \pmod n$, thereby deriving the factor $\gcd(X-Y, n)$ of $n$.

However, the essence of GNFS is that it does not do this only in the rational number field $\mathbb{Q}$, but simultaneously searches for “smooth numbers” in both an extension field called an Algebraic Number Field $\mathbb{Q}(\alpha)$ and the rational number field, building congruence relations through homomorphisms.

The GNFS process is broadly divided into five phases.

graph TD A["Integer Factorization Problem (Input n)"] --> B["1. Polynomial Selection"] B --> C["2. Sieving Phase"] C --> D["3. Filtering Phase"] D --> E["4. Linear Algebra Phase"] E --> F["5. Square Root Phase"] F --> G["Output prime factors p, q"]

3.1 Phase 1: Polynomial Selection

The success of GNFS heavily depends on the selection of appropriate polynomials. The goal is to find two irreducible polynomials $f_1(x)$ (rational side) and $f_2(x)$ (algebraic side) that share a common root $m$. That is, it satisfies: $f_1(m) \equiv f_2(m) \equiv 0 \pmod n$

Usually, a polynomial of degree 1 is chosen for the rational side, $f_1(x) = x - m$, and a monic polynomial of degree $d$ (typically 5 or 6) is chosen for the algebraic side, $f_2(x)$. The most classical approach is the Base-$m$ method. Choose an integer $m = \lfloor n^{1/(d+1)} \rfloor$ close to the $1/(d+1)$ power of $n$, and expand $n$ in base $m$. $n = c_d m^d + c_{d-1} m^{d-1} + \dots + c_1 m + c_0$ This gives the polynomial $f_2(x) = c_d x^d + c_{d-1} x^{d-1} + \dots + c_0$. Obviously, $f_2(m) = n \equiv 0 \pmod n$.

However, in modern implementations, Kleinjung’s algorithm is used. This optimizes algebraic properties (Murphy’s $E$ value and $\alpha$-value) while preventing the coefficients of the polynomial from becoming extremely large (optimizing skewness), exploring polynomials that are likely to generate smooth numbers during the sieving phase. A massive amount of computational resources is invested in this step alone.

3.2 Phase 2: Sieving Phase

Once the polynomials are determined, the algorithm enters the “Sieving” phase, which has the highest computational load. Here, we search for pairs $(a, b)$. This pair is coprime, and the following two values are simultaneously required to be “smooth”.

Norm on the rational side: $F_1(a, b) = b \cdot f_1(a/b) = a - bm$
Norm on the algebraic side: $F_2(a, b) = b^d \cdot f_2(a/b)$

“Smooth” means that it can be factored only by primes up to a specified limit (Sieve bound). A prime base (Factor base) for the rational side and a prime base for the algebraic side are prepared, and smooth numbers are efficiently found over a huge search space using an approach similar to the Sieve of Eratosthenes. Currently, a method called Lattice Sieving is mainstream. By fixing a specific prime $q$ and sieving only the $(a, b)$ pairs on a sublattice where both the rational side and the algebraic side become multiples of $q$, extremely high efficiency is realized.

3.3 Phase 3: Filtering Phase

The number of smooth relations found in the sieving phase reaches hundreds of millions to billions. However, these also contain a lot of useless information. The purpose of filtering is to construct a huge sparse matrix while reducing its dimensions as much as possible.

Specifically, operations such as the following are performed.

Singleton removal: Remove relations that contain a prime factor that appears only once.
Clique removal / Merging: Multiply relations that share prime factors appearing two or more times, eliminating variables and reducing to a denser but smaller-dimensional system of equations.

As a result, a matrix with billions of rows is compressed into a huge sparse matrix $\mathbf{A}$ with tens of millions of rows (elements are 0 and 1 over the field $\mathbb{F}_2$).

3.4 Phase 4: Linear Algebra Phase

Here, we find a non-trivial solution vector $\mathbf{x}$ for the equation $\mathbf{A} \mathbf{x} \equiv \mathbf{0} \pmod 2$. In other words, this is the problem of finding the left nullspace of a huge sparse matrix.

Because the matrix size is extremely large, normal Gaussian elimination ($O(N^3)$) is completely impossible to compute. Therefore, an iterative method, a type of Krylov subspace method, is used. Historically, the Block Lanczos method has been used, but in modern distributed computing environments, the Block Wiedemann Algorithm, which can dramatically reduce communication overhead, is mainstream.

The Block Wiedemann method calculates the minimal polynomial from the matrix $\mathbf{A}$ and a sequence of vectors, and constructs the basis of the nullspace using the Berlekamp-Massey algorithm. This step is extremely difficult to parallelize, and is one of the biggest bottlenecks of GNFS, requiring a tightly coupled communication network of supercomputers or large-scale clusters.

3.5 Phase 5: Square Root Phase

From the solution of linear algebra, a product that becomes a “perfect square” is constructed on each of the rational and algebraic sides. On the rational side, $\prod (a-bm)$ becomes the square $X^2$ of some integer $X$, and on the algebraic side, the corresponding product of ideals becomes a perfect square $\gamma^2$ over the algebraic field. By computing this $\gamma$ over the algebraic field and applying the homomorphism $\phi: \alpha \mapsto m \pmod n$ to the ring of rational integers, the congruence: $X^2 \equiv \phi(\gamma)^2 \equiv Y^2 \pmod n$ is obtained.

Computing the square root over the algebraic field requires deep knowledge of algebraic number theory, using complex algorithms such as Montgomery’s Method. Finally, $\gcd(X-Y, n)$ is calculated, and if a non-trivial factor is obtained, the factorization is complete.

4. Are There Classical Algorithms Beyond GNFS?

To date, no classical algorithm has been discovered whose asymptotic complexity falls below $L_n[1/3, c]$ for the factorization of general integers. However, there are some attempts and derivative algorithms to break through theoretical and practical limits.

4.1 Multiple Number Field Sieve (MNFS)

As an approach extending GNFS, there is the Multiple Number Field Sieve (MNFS) by D. Coppersmith. While GNFS uses two polynomials (rational side and algebraic side), MNFS uses multiple different algebraic side polynomials simultaneously for a single rational side polynomial.

$$ f_1(x), f_{2,1}(x), f_{2,2}(x), \dots, f_{2,V}(x) $$

By utilizing multiple algebraic fields, the probability of “becoming smooth in any of the algebraic fields” can be dramatically increased in each sieving step. Coppersmith succeeded in slightly reducing the constant $c$ in the complexity $L_n[1/3, c]$ through this approach. Specifically, while the constant of GNFS is $c = (64/9)^{1/3} \approx 1.923$, it has been theoretically shown that optimizing MNFS can reduce the complexity to about $c \approx 1.902$. However, in practice, the overhead of managing multiple fields is large, and it has not yet led to a decisive breakthrough for RSA moduli on a practical scale.

4.2 Is an $L_n[1/4]$ Class Algorithm Possible?

Regarding the limits of classical integer factorization algorithms, a theme that has been debated among mathematicians for many years is the question, “Does an algorithm with an exponent $\alpha = 1/4$ exist?” Current GNFS and its derivatives are strongly bound to the framework of “searching for smoothness” by sieving, and within this paradigm, it is widely believed that $\alpha = 1/3$ is the limit. Even from the analysis of the distribution probability of smooth integers using the Dickman function, it is thought that with the current combination of algebraic field construction methods and sieves, the barrier of $O(L_n[1/3])$ cannot be crossed no matter how much it is optimized.

If an $L_n[1/4]$ or even a classical polynomial-time algorithm were to exist, it would have to rely on entirely new mathematical structures that humanity currently cannot conceive of (for example, a more advanced algebraic geometry approach like Schoof’s algorithm for elliptic curve cryptography), completely different from the “smoothness-based” approach like GNFS. However, there are no signs of such at present.

5. Breakthrough by Quantum Computers: Shor’s Algorithm

While classical computers face the barrier of $L_n[1/3]$, Shor’s Algorithm, published by Peter Shor in 1994, smashed this barrier by fundamentally changing the computation model itself.

5.1 The Impact of Quantum Polynomial Time

Shor’s algorithm reduces the integer factorization problem to the “Order Finding Problem”. For a certain integer $a$, it is the problem of finding the period (order) $r$ of the function $f(x) = a^x \pmod n$. While classical computers require exponential time to find this period, by using Quantum Phase Estimation (QPE) and the Quantum Fourier Transform (QFT) on a quantum computer, it is possible to evaluate all superpositions of states in parallel and extract the period $r$ with high probability.

In terms of computational complexity, the execution time of Shor’s algorithm is quantum polynomial time, specifically as follows:

$$ O((\log n)^3) $$

Considering recent optimized circuit implementations, it is said that it can be reduced to $O((\log n)^2 \log \log n)$.

graph LR A["Classical Algorithm (GNFS)"] -->|Limit| B["Sub-exponential Time L_n[1/3]"] C["Quantum Algorithm (Shor)"] -->|Breakthrough| D["Polynomial Time O((log n)^3)"] B --> E["Continued use of RSA (Increase key length)"] D --> F["Complete collapse of RSA cryptography"]

5.2 Classical Sub-exponential Time vs. Quantum Polynomial Time

The difference between these two complexity classes holds decisive meaning in real-world cryptographic security.

For example, consider the case of factoring RSA-2048 (a 2048-bit composite number).

GNFS (Classical): Substituting $n \approx 2^{2048}$ into $L_n[1/3, 1.923]$, about $2^{112}$ operations are required. This is an astronomical amount of computation that would take longer than the lifespan of the universe even if all the computing resources on Earth today were mobilized.
Shor’s Algorithm (Quantum): With an $O((\log n)^3)$ algorithm, about $2048^3 \approx 8.5 \times 10^9$ logical gate operations are sufficient. This means that if appropriate hardware (a universal quantum computer with millions of physical qubits and error correction capabilities) exists, the calculation could be completed in just a few hours to a few days.

The paradigm shift from the sub-exponential function of “exponent $\alpha=1/3$” to “polynomial time” neutralizes the traditional cryptographic strategy of ensuring security by increasing the key length.

6. Conclusion: Outlook for the Next Generation

The current scientific consensus on the question “Are there classical algorithms beyond GNFS?” is as follows:

Practical improvements continue, but there are no asymptotic leaps: Attempts to improve the constant term $c$ of GNFS, such as MNFS, optimization of polynomial selection, and parallelization of the Block Wiedemann method, are ongoing. However, the possibility of discovering a classical algorithm with $\alpha$ falling below $1/3$ is considered extremely low.
The security of RSA on classical computers remains strong: The computational complexity of GNFS remains enormous, and RSA-2048 and RSA-4096 will continue to maintain their security against attacks by classical computers for decades to come.
The true threat is quantum algorithms: What crossed the barrier of computational complexity was Shor’s algorithm, based on the principles of quantum mechanics. As a result, the world is forced to transition to Post-Quantum Cryptography (PQC). The transition to new mathematical problems that are considered difficult to solve even for quantum computers (cannot be solved in polynomial time), such as lattice-based cryptography and hash-based cryptography, is currently at the forefront of cryptography.

The General Number Field Sieve (GNFS) is one of the “highest peaks” humanity has reached by challenging the limits of classical mathematics and algorithm design. Understanding the profound mathematical structure of GNFS is not merely learning the history of cryptanalysis, but also an intellectual journey of exploration that touches upon the beauty of computational complexity theory and algebraic number theory. Until the day quantum computers are put into practical use, GNFS will likely continue to defend its throne as the strongest integer factorization algorithm.

Simulating Shor's Algorithm in Python

Fri, 11 Sep 2026 08:00:00 +0900

1. Introduction: The Cryptographic Crisis Brought by Quantum Computers

Much of the security in modern internet society relies on public-key cryptography (especially RSA encryption). When we transmit credit card information for online shopping or exchange highly confidential data, the content of that communication is strongly protected by RSA encryption.

The basis for the security of RSA encryption relies on the mathematical fact that “factoring huge integers is extremely difficult for classical computers (the PCs and supercomputers we use every day).” However, Shor’s Algorithm, published by Peter Shor in 1994, fundamentally overturned this premise. It was mathematically proven that if Shor’s algorithm were executed on a large-scale quantum computer, it could solve factorization problems—which would take classical computers longer than the age of the universe—in just minutes to hours.

In this article, we will thoroughly explain how Shor’s algorithm performs integer factorization so quickly in detail, from its mathematical mechanics to a concrete simulation implementation using Python and the quantum computing framework Qiskit.

2. Dramatic Shift in Computational Complexity: From Exponential to Polynomial Time

Why is prime factorization so difficult? Even if we use the “General Number Field Sieve (GNFS),” known as the best factorization algorithm for classical computers, its computational complexity is sub-exponential.

The time complexity to factorize a composite number of $N$ digits using classical methods is as follows:

$$ O\left(\exp\left( c (\log N)^{1/3} (\log \log N)^{2/3} \right)\right) $$

Because of this, simply increasing the key length (e.g., to 2048 bits or 4096 bits) ensures that deciphering it on a classical computer would take thousands or tens of thousands of years—an unrealistic amount of time.

However, using Shor’s Algorithm on a quantum computer dramatically reduces the computational complexity to polynomial time relative to the number of input bits $\log N$:

$$ O((\log N)^3) $$

This means that if we double the number of bits, the computation time on a classical computer increases astronomically, whereas on a quantum computer it only increases by at most about 8 times. This reduction in complexity class from exponential time to polynomial time (inclusion in the BQP class) is the true marvel of Shor’s algorithm.

graph TD A["Increase in input size (bits) N"] --> B{"Algorithm selection"} B -->|Classical: General Number Field Sieve| C["Sub-exponential increase O(exp(...))"] B -->|Quantum: Shor's algorithm| D["Polynomial time O((log N)^3)"] C --> E["Thousands to billions of years (Undecipherable)"] D --> F["Minutes to hours (Decipherable in realistic time)"]

3. Algorithm Overview and Mathematical Background

Shor’s algorithm does not actually perform everything on a quantum computer. It is composed of a collaboration between pre-processing and post-processing on a classical computer and the core part (the order-finding algorithm) on a quantum computer.

The overall flow of the algorithm is as follows:

graph TD A["Input: Composite number N to be factored"] --> B["Choose a random number a such that a < N"] B --> C{"gcd(a, N) > 1 ?"} C -- "Yes" --> D["Output trivial factor gcd(a, N) and terminate"] C -- "No" --> E["Find the period r of f(x) = a^x mod N using quantum algorithm"] E --> F{"Is r even AND a^(r/2) ≢ -1 mod N ?"} F -- "No" --> B F -- "Yes" --> G["Calculate factors p = gcd(a^(r/2) - 1, N), q = gcd(a^(r/2) + 1, N)"] G --> H["Output: p, q"]

Reduction of Factorization to the Order Finding Problem

Shor’s stroke of genius lies in converting the “factorization problem” into an “Order Finding Problem”.

Consider an integer $N$ (the number to be factored) and an integer $a$ coprime to $N$ ($1 < a < N$). We define the following modular exponentiation function:

$$ f(x) = a^x \bmod N $$

This function has a certain period $r$. That is, $f(x+r) = f(x)$ holds true for any $x$. In particular, when $x=0$, the smallest positive integer $r$ such that:

$$ a^r \equiv 1 \pmod N $$

is called the “order of $a$ modulo $N$”. If we can find this period $r$, we can derive the prime factors as follows.

Rearranging the equation gives:

$$ a^r - 1 \equiv 0 \pmod N $$

If $r$ is even, we can factor it using the difference of squares formula:

$$ (a^{r/2} - 1)(a^{r/2} + 1) \equiv 0 \pmod N $$

This means that $N$ shares a common divisor with either $(a^{r/2} - 1)$ or $(a^{r/2} + 1)$ (provided that the condition $a^{r/2} \not\equiv -1 \pmod N$ is met). Therefore, using the Euclidean algorithm to calculate:

$$ p = \gcd(a^{r/2} - 1, N) $$ $$ q = \gcd(a^{r/2} + 1, N) $$

allows us to find the non-trivial prime factors $p, q$ of $N$. This computation (calculating the greatest common divisor and generating random numbers) can be done extremely fast on classical computers. The problem is thus narrowed down to how to find the period $r$ quickly. On a classical computer, finding this period $r$ itself takes exponential time. This is where the quantum computer comes into play.

4. Quantum Algorithm Part: Mechanics of Order Finding

The subroutine for finding the period $r$ using a quantum computer consists of the following 4 steps:

graph LR subgraph "Quantum State Transitions" S1["|0⟩|0⟩ (Initialization)"] --> S2["H Gate: Superposition Σ|x⟩|0⟩"] S2 --> S3["Oracle U: Σ|x⟩|a^x mod N⟩"] S3 --> S4["QFT: Period extraction via interference"] S4 --> S5["Measurement: Obtain approximated value y"] end

Step 1: Initialization of Quantum Registers and Superposition

First, we prepare two quantum registers. The first register is for inputting states, and the second register is for storing the result of the function’s calculation. The initial state is all $|0\rangle$.

$$ |\psi_0\rangle = |0\rangle_1 |0\rangle_2 $$

We apply Hadamard gates to all qubits in the first register, creating an equal-probability superposition of all possible inputs $x$ (from $0$ to $Q-1$, where $Q=2^n$).

$$ |\psi_1\rangle = \frac{1}{\sqrt{Q}} \sum_{x=0}^{Q-1} |x\rangle_1 |0\rangle_2 $$

Through this, the quantum computer simultaneously holds the states for all $Q$ inputs in a single operation. This is the powerful source of quantum parallelism.

Step 2: Application of the Oracle Function (Modular Exponentiation)

Next, using a quantum arithmetic circuit $U_f$, we calculate the function $f(x) = a^x \bmod N$ and store the result in the second register.

$$ |\psi_2\rangle = \frac{1}{\sqrt{Q}} \sum_{x=0}^{Q-1} |x\rangle_1 |a^x \bmod N\rangle_2 $$

At this point, the first and second registers are in a state of quantum entanglement. If we were to (hypothetically) observe the second register and obtain a specific value $k = a^{x_0} \bmod N$, the state of the first register would collapse into a superposition of $x$ values that yield that $k$. Since the period of the function is $r$, the remaining states will be values separated by $r$: $x_0, x_0+r, x_0+2r, \dots$

$$ |\psi_3\rangle = \sqrt{\frac{r}{Q}} \sum_{j=0}^{M-1} |x_0 + j r\rangle_1 |k\rangle_2 $$

However, we do not want to know $x_0$; we want to know the period $r$ itself. It is impossible to observe $r$ directly from this state. Therefore, we use the Quantum Fourier Transform.

Step 3: Phase Interference via Quantum Fourier Transform (QFT)

We apply the Quantum Fourier Transform (QFT) to the first register. QFT is the quantum version of the classical discrete Fourier transform, and it transforms the amplitudes of the state vector. The action of QFT on the basis state $|x\rangle$ is defined as follows:

$$ QFT |x\rangle = \frac{1}{\sqrt{Q}} \sum_{y=0}^{Q-1} \omega^{xy} |y\rangle $$

Here, $\omega = e^{2\pi i / Q}$.

When QFT is applied, the state amplitudes interfere with each other. Skipping the mathematical details, when QFT is applied to a state with period $r$, the waves cause Constructive Interference only when $y$ is extremely close to an integer multiple of $Q/r$. For all other states, the probability amplitudes cancel out due to Destructive Interference, approaching zero.

Step 4: Measurement and Continued Fraction Expansion

Finally, we measure the first register. The value $y$ obtained from the measurement will satisfy the following condition with high probability:

$$ y \approx c \frac{Q}{r} \implies \frac{y}{Q} \approx \frac{c}{r} $$

(where $c$ is an unknown integer such that $0 \le c < r$)

By applying the classical algorithm of Continued Fraction Expansion to the obtained rational number $y/Q$, we can calculate the approximated fraction $c/r$ and extract the period $r$ from its denominator.

5. Simulation Implementation using Python and Qiskit

Since theory alone can be hard to grasp, let’s actually simulate Shor’s algorithm using Python and IBM’s quantum computing framework, Qiskit.

Here, we will implement the most classic and famous example scenario: “Factoring $N=15$ using $a=7$.”

Preparation of the Execution Environment

Please install Qiskit beforehand.

1

pip install qiskit qiskit-aer numpy

Overview of the Python Implementation Code

The code below is an example implementation of Shor’s algorithm specialized for $N=15, a=7$. Because building a general-purpose modular exponentiation circuit is currently too computationally expensive for simulators, we are hardcoding the gate operations for the specific case of $a=7$.

 1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123


import numpy as np
from qiskit import QuantumCircuit
from qiskit_aer import AerSimulator
from qiskit.visualization import plot_histogram
from fractions import Fraction
import math

# 1. Function to build the inverse Quantum Fourier Transform (QFT†)
def qft_dagger(n):
 """Generates an n-qubit inverse Quantum Fourier Transform circuit"""
 qc = QuantumCircuit(n)
 # SWAP gates to reverse the order
 for qubit in range(n//2):
 qc.swap(qubit, n-qubit-1)
 # Applying controlled-phase and H gates
 for j in range(n):
 for m in range(j):
 qc.cp(-np.pi/float(2**(j-m)), m, j)
 qc.h(j)
 qc.name = "QFT_dagger"
 return qc

# 2. Function to build the controlled modular exponentiation for 7^x mod 15
def c_amod15(a, power):
 """Generates a controlled U gate for specific a and power (N=15 only)"""
 U = QuantumCircuit(4)
 for _ in range(power):
 # Hardcoded logic for 7^x mod 15 when a=7
 if a in [2,13]:
 U.swap(2,3)
 U.swap(1,2)
 U.swap(0,1)
 if a in [7,8]:
 U.swap(0,1)
 U.swap(1,2)
 U.swap(2,3)
 if a in [4, 11]:
 U.swap(1,3)
 U.swap(0,2)
 if a in [7,11,13]:
 for q in range(4):
 U.x(q)
 U = U.to_gate()
 U.name = f"{a}^{power} mod 15"
 c_U = U.control()
 return c_U

# 3. Main quantum circuit configuration
def shor_circuit(a, n_count):
 # n_count: Number of qubits in the control register
 # The target register is 4 bits to represent 0 to 15
 qc = QuantumCircuit(n_count + 4, n_count)

 # Initialization of the 1st register (control register) (Generating superposition)
 for q in range(n_count):
 qc.h(q)

 # Initialization of the 2nd register (target register) to |1> (0001)
 qc.x(3 + n_count)

 # Application of the controlled modular exponentiation (oracle)
 for q in range(n_count):
 # Apply the operation for 2^q
 qc.append(c_amod15(a, 2**q),
 [q] + [i+n_count for i in range(4)])

 # Apply inverse quantum Fourier transform to the 1st register
 qc.append(qft_dagger(n_count), range(n_count))

 # Measure the 1st register
 qc.measure(range(n_count), range(n_count))
 return qc

# --- Execution Section ---
if __name__ == "__main__":
 N = 15
 a = 7
 n_count = 8 # Use 8 qubits for the control register (Q=256)

 print(f"Search settings: N={N}, a={a}, Control qubits={n_count}")

 # Generate the circuit
 qc = shor_circuit(a, n_count)

 # Execute on the simulator
 sim = AerSimulator()
 # transpilation is recommended in newer Qiskit versions
 from qiskit import transpile
 compiled_circuit = transpile(qc, sim)
 job = sim.run(compiled_circuit, shots=1024)
 result = job.result()
 counts = result.get_counts()

 print("\nMeasurement results (bitstring: observation count):")
 for bitstring, count in counts.items():
 print(f" {bitstring}: {count} times")

 # Classical post-processing: Identifying period r using continued fraction expansion
 print("\n--- Period calculation and factorization ---")
 phases = []
 for output in counts:
 # Convert bitstring to decimal
 decimal = int(output, 2)
 # Phase = measurement value / 2^n_count
 phase = decimal / (2**n_count)
 phases.append(phase)

 # Get approximated fraction using continued fraction expansion. Denominator limit is N=15
 frac = Fraction(phase).limit_denominator(15)
 r = frac.denominator

 print(f"Observation: {decimal:3d} | Phase: {phase:.4f} | Continued fraction: {frac} | Estimated period r = {r}")

 # Check if period r is even and yields valid results
 if r % 2 == 0:
 guess1 = math.gcd(a**(r//2) - 1, N)
 guess2 = math.gcd(a**(r//2) + 1, N)
 if guess1 not in [1, N] or guess2 not in [1, N]:
 print(f" => Success! The prime factors of {N} are {guess1} and {guess2}.")
 else:
 print(f" => Only trivial factors found. Try again.")
 else:
 print(f" => Failed because the period is odd.")

Code Explanation and Result Analysis

When executing the above code, specific peaks (observed values) are obtained with high probability as the measurement results of the control register. In the case of n_count=8 ($Q=256$), with an ideal quantum computer (or simulator), values like 0, 64, 128, and 192 will appear as observed values with overwhelming probability.

Dividing these by $Q=256$, the phase $y/Q$ becomes $0.0$, $0.25$, $0.5$, and $0.75$, respectively. Applying continued fraction expansion to these phases yields:

$0.25 \to 1/4$ (Estimated period $r=4$)
$0.50 \to 1/2$ (Estimated period $r=2$)
$0.75 \to 3/4$ (Estimated period $r=4$)

Using the period $r=4$ obtained here, we calculate the prime factors. Since $a=7, r=4$: $p = \gcd(7^2 - 1, 15) = \gcd(48, 15) = 3$ $q = \gcd(7^2 + 1, 15) = \gcd(50, 15) = 5$

Brilliantly, we have successfully factorized $15 = 3 \times 5$.

[!TIP] If a measurement value of $y=128$ (phase $0.5$) is obtained, the denominator becomes $2$, yielding a divisor of the true period rather than the true period $r=4$. In such cases, the true period can be reached by either executing the algorithm multiple times or by investigating multiples of the obtained $r$.

6. Challenges Toward Practical Application and the Limits of the NISQ Era

While it was easy to factorize $N=15$ on a simulator, factoring RSA-2048 (a 617-digit decimal number) used in real-world applications still faces numerous walls for actual quantum computers.

The era we are currently living in is called the NISQ (Noisy Intermediate-Scale Quantum) era. Qubits are extremely vulnerable to noise from the external environment, and their states break down midway through computations due to “decoherence.”

In order to accurately execute deep circuits (with many gates) like Shor’s algorithm, Quantum Error Correction to correct noise is essential. To create a single noise-free “logical qubit,” it is necessary to encode thousands of “physical qubits” using methods like the Surface Code.

To break 2048-bit RSA encryption, it is estimated that thousands of perfect logical qubits are required, and realizing this would necessitate a fault-tolerant quantum computer equipped with millions to tens of millions of physical qubits. Since even the most advanced current quantum processors only have around a few hundred to a few thousand physical qubits, the world’s cryptography will not be broken immediately.

[!WARNING] However, there exists a threat model known as “Store Now, Decrypt Later”. Attackers might store large amounts of currently encrypted confidential communication data, adopting a strategy to decrypt everything all at once 10 to 20 years later the moment a powerful quantum computer is completed.

7. Transitioning to Post-Quantum Cryptography (PQC)

In preparation for the arrival of “Q-Day” (the day quantum computers break cryptography), cryptographers around the world, spearheaded by the National Institute of Standards and Technology (NIST) in the US, are pushing forward with the standardization of Post-Quantum Cryptography (PQC).

PQC is based on new mathematical problems (such as lattice problems, multivariate polynomial problems, and hash-based functions) that are mathematically considered inefficient to solve even using Shor’s algorithm (or Grover’s algorithm). Algorithms like “CRYSTALS-Kyber” and “CRYSTALS-Dilithium” have already been selected as standard specifications, and their integration into Apple’s iMessage and various web browser communication protocols is gradually beginning.

For engineers managing IT infrastructure, building “crypto-agility” (the ability to quickly switch cryptographic methods) into systems to transition from existing RSA or elliptic curve cryptography to PQC will be a major mission going forward.

8. Conclusion

In this article, we provided a thorough explanation on a scale of 10,000 characters, starting from the theoretical mathematical background of Shor’s algorithm, covering the mechanics of period extraction using the Quantum Fourier Transform, and ending with a concrete simulation code using Python and Qiskit.

The fact that the laws of physics in the microscopic world of quantum mechanics can fundamentally overturn computational complexity theory and cryptography—which are the foundations of macroscopic information science—is one of the most exciting paradigm shifts in the history of science. We must keep a close eye on the ongoing offensive and defensive battle between the continuously evolving quantum computing technology and the new cryptographic techniques standing up against it.

By all means, please try executing the Python code introduced this time in your own environment, and experience the “magic of computation” created by the superposition and interference of quantum states.

References

Shor, P. W. (1994). “Algorithms for quantum computation: discrete logarithms and factoring”. Proceedings 35th Annual Symposium on Foundations of Computer Science.
Nielsen, M. A., & Chuang, I. L. (2010). “Quantum Computation and Quantum Information”. Cambridge University Press.
Qiskit Documentation: https://qiskit.org/documentation/

[Illustrated PQC] Comparison of Major Post-Quantum Cryptography Algorithms

Fri, 11 Sep 2026 07:00:00 +0900

1. Introduction: The “Cryptography Crisis” Brought by Quantum Computers

In modern internet society, public-key cryptography is indispensable infrastructure for protecting the confidentiality of communications and data integrity. The widely used RSA cryptography and Elliptic Curve Cryptography (ECC) rely on the mathematical barriers of “the difficulty of factoring large composite numbers” and “the difficulty of the discrete logarithm problem on elliptic curves,” respectively. It has been proven that classical computers (including the supercomputers we use today) would take longer than the age of the universe to solve these mathematical problems, which has been the basis of their security.

However, this solid premise is about to be completely overturned by the theory and practical advancement of quantum computers. The “Shor’s Algorithm”, published by cryptographer Peter Shor in 1994, theoretically proved that the integer factorization problem and the discrete logarithm problem can be solved in “polynomial time” by running it on a sufficiently capable Cryptographically Relevant Quantum Computer (CRQC). This means that all public-key cryptography currently in use will be rendered powerless.

graph TD A["Large-scale Quantum Computer (CRQC)"] -->|Execution| B["Shor's Algorithm"] B -->|Decryption in polynomial time| C["Integer Factorization Problem (RSA)"] B -->|Decryption in polynomial time| D["Discrete Logarithm Problem (ECC / ECDSA)"] C --> E["Eavesdropping, data tampering, and spoofing of encrypted communications"] D --> E F["Store Now, Decrypt Later (SNDL)"] --> E

It is extremely dangerous to think that “there is no problem because the full-scale completion of quantum computers is still decades away.” This is because an attack method called Store Now, Decrypt Later (SNDL) is already a real threat. This is an attack where malicious states or hacker organizations save a massive amount of currently encrypted communication data (such as TLS traffic) in storage and decrypt all of it the moment a powerful quantum computer becomes available in the future. State secrets, infrastructure information, and medical data that need long-term protection are already exposed to this threat.

Furthermore, for symmetric-key cryptography (such as AES) and hash functions (such as SHA-256), there is Grover’s Algorithm, discovered in 1996. This reduces the computational complexity of a brute-force attack to its square root. In other words, the security level of AES-128 is effectively halved to $2^{64}$, so it is recommended to use longer keys and hash lengths, such as AES-256 and SHA-384, in the quantum era.

To counter this unprecedented cryptography crisis, Post-Quantum Cryptography (PQC) was born, which is based on new mathematical problems that are difficult to decrypt even with a quantum computer. This article provides an extremely detailed explanation of the major PQC algorithms, from their mathematical background to their mechanisms and architectural comparisons, based on the results of the PQC standardization process led by the National Institute of Standards and Technology (NIST) in the United States.

2. Overview and History of the NIST PQC Standardization Project

Transitioning cryptographic technologies takes years to decades, including redesigning protocols, updating systems, and replacing hardware. Therefore, cryptographers around the world have been advancing PQC research since early on. The US NIST (National Institute of Standards and Technology) has played a central role in this. In 2016, NIST announced a public call for the PQC standardization process and accepted entirely new cryptographic algorithm proposals from the global cryptographic community.

The targets for standardization were the following two main categories:

Public-Key Cryptography / Key Encapsulation Mechanism (KEM): A mechanism for securely sharing (distributing) a shared key to encrypt the communication path, such as in TLS connections.
Digital Signatures: A mechanism to prove that data has not been tampered with and that there is no spoofing of the sender (authenticity) in software updates and electronic certificates.

After a fierce competition of evaluation, analysis, and cryptanalysis spanning about 6 years (Round 1 to Round 3), further evaluation for Round 4 was conducted for some algorithms. As a result, the following algorithms were officially published as Federal Information Processing Standards (FIPS) in 2024 and established as the future global standards:

FIPS 203 (ML-KEM): KEM based on CRYSTALS-Kyber
FIPS 204 (ML-DSA): Digital signature based on CRYSTALS-Dilithium
FIPS 205 (SLH-DSA): Stateless hash-based signature based on SPHINCS+
(Scheduled for future formulation) FN-DSA: Digital signature based on FALCON

These selected algorithms rely on different mathematical “hardness problems,” ensuring diversity (Crypto Agility) so that even if a fatal vulnerability is discovered in one algorithm in the future, the entire system will not collapse. In the standardization process, lattice-based cryptography became the main player mainly due to its performance, but hash-based cryptography and code-based cryptography were adopted as powerful backups.

3. Classification of Major Mathematical Approaches in PQC

PQC algorithms are broadly divided into the following five categories based on the mathematical problems that form the basis of their security. This article delves deeply into the top three in particular.

Lattice-based Cryptography: Based on the Shortest Vector Problem (SVP) and Closest Vector Problem (CVP) in multidimensional lattice spaces, and the derived LWE problem. It is the center of NIST standardization, and Kyber, Dilithium, and FALCON fall into this category. It has the best balance of processing speed, public key size, and ciphertext size, making it suitable for general-purpose use.
Hash-based Cryptography: Relies solely on the “collision resistance” and “one-wayness” of cryptographic hash functions (such as SHA-2 and SHAKE) for its security basis. It is only applicable to digital signatures (such as SPHINCS+), but its security proof is the strongest, and it features extremely high resistance to unknown mathematical attacks.
Code-based Cryptography: Based on the theory of error-correcting codes, it relies on the difficulty of the Syndrome Decoding Problem. Classic McEliece, proposed in the 1970s, is a representative example, having a very long history and proven security, but on the other hand, the public key size is extremely large, in the megabyte range.
Multivariate Polynomial Cryptography: Based on the difficulty of finding a solution to a system of multivariate quadratic equations over a finite field (MQ problem). It was mainly proposed as digital signatures (such as Rainbow), but during the final round of NIST, a powerful attack method that could crack it in a few days on a single PC was discovered, and many algorithms dropped out of the standardization.
Isogeny-based Cryptography: Based on the path-finding problem on an isogeny graph of elliptic curves. The key size is very small, and it was expected to be a legitimate successor to ECC, but “SIKE,” the final candidate, was completely broken in just a few hours on a normal PC in 2022 using classical mathematics (such as the Castryck-Decru attack), marking a dramatic end that symbolized the difficulty and terror of PQC design.

4. The Abyss of Lattice Cryptography: Mathematical Foundation of the LWE Problem and Module-LWE

Lattice-based cryptography is currently considered the most promising and has become the center of standardization. At the root of its security is the LWE (Learning with Errors) problem. Proposed by Oded Regev in 2005, this groundbreaking achievement earned him the Gödel Prize. One cannot talk about modern PQC without understanding the LWE problem.

4.1. What is the LWE (Learning with Errors) Problem?

First, consider a simple system of linear equations. Suppose there is a known random matrix $A$ and an unknown secret vector $\vec{s}$ under a certain modulus $q$ (modulo $q$), and their product $\vec{b}$ is given:

$$ \vec{b} = A\vec{s} \pmod q $$

In this case, it is easy to find the unknown $\vec{s}$ from the public information $A$ and $\vec{b}$. Using the classical algorithm “Gaussian elimination,” $\vec{s}$ can be easily calculated in polynomial time.

However, adding a “small intentional error (noise)” to this equation dramatically increases the difficulty of the problem. This is the LWE problem.

Prepare an unknown secret vector $\vec{s} \in \mathbb{Z}_q^n$ and a randomly chosen matrix $A \in \mathbb{Z}_q^{m \times n}$. Furthermore, prepare an error vector $\vec{e} \in \mathbb{Z}_q^m$ whose “elements have sufficiently small values,” chosen according to a normal or binomial distribution, and calculate $\vec{b}$ as follows:

$$ \vec{b} = A\vec{s} + \vec{e} \pmod q $$

The Search LWE problem is the problem of “finding the secret information $\vec{s}$ from the public information $(A, \vec{b})$.” Due to the existence of this error $\vec{e}$, if one attempts an algebraic solution such as Gaussian elimination, the error $\vec{e}$ amplifies like a snowball in the process of adding and subtracting equations, ultimately becoming indistinguishable from random values and breaking down.

The greatness of the LWE problem lies in the fact that there is a powerful theoretical proof (reduction) that unless there is a quantum algorithm that can solve GapSVP (Decision Shortest Vector Problem) and SIVP (Shortest Independent Vector Problem), which are “worst-case hardness” problems on lattices, the LWE problem cannot be solved in the average-case either. In other words, even for a randomly generated cryptographic key, it is guaranteed to have robust security backed by a theoretical upper bound.

4.2. Dramatic Efficiency Improvement by Ring-LWE and Module-LWE

The normal LWE problem (Standard LWE) has a very clear basis for security, but the size of the matrix $A$ becomes very large, and the key size reaches the megabyte class, making it impractical. Therefore, an approach was proposed to provide an algebraic structure by utilizing Polynomial Rings.

In the Ring-LWE problem, instead of simple vectors and matrices, elements (polynomials) of a certain polynomial ring $R_q$ are used. The following cyclotomic polynomial ring is generally used in the NIST standard:

$$ R_q = \mathbb{Z}_q[X]/(X^n + 1) $$

Here, $n$ is a power of 2 (e.g., 256), and $q$ is an appropriate prime number. Over this ring, using elements $a, s, e \in R_q$, $b = a \cdot s + e \pmod q$ is calculated. Because a single polynomial $a$ has $n$ coefficients, data can be significantly compressed, and by using the finite field version of the Fast Fourier Transform (FFT) called NTT (Number Theoretic Transform), ultra-fast polynomial multiplication becomes possible with a computational complexity of $O(n \log n)$.

However, Ring-LWE had concerns that “there might be an unknown vulnerability due to the special algebraic structure of the ring.” Furthermore, there was an engineering challenge that when changing the security level (such as AES-128, 192, 256 equivalent), the degree $n$ of the polynomial itself had to be changed, and the entire implementation, such as the NTT algorithm, had to be rewritten accordingly.

Therefore, the Module-LWE (M-LWE) problem was adopted by standardization algorithms such as Kyber and Dilithium. Module-LWE is a compromise situated exactly halfway between the structureless Standard LWE and the overly structured Ring-LWE, using a $k \times k$ matrix (module) whose components are elements of the polynomial ring $R_q$:

$$ \vec{b} = A\vec{s} + \vec{e} \pmod{R_q} \quad (A \in R_q^{k \times k}, \vec{s}, \vec{e} \in R_q^k) $$

The greatest advantage of Module-LWE is that the security level can be easily scaled simply by changing the matrix dimension $k$ while keeping the polynomial degree $n$ (in the NIST standard, $n=256$) fixed. For example, in Kyber, the dimension $k$ is adjusted as follows:

Kyber512 (Level 1): $k = 2$ (equivalent to AES-128)
Kyber768 (Level 3): $k = 3$ (equivalent to AES-192)
Kyber1024 (Level 5): $k = 4$ (equivalent to AES-256)

This made it possible to reuse 100% of the underlying NTT code and polynomial operation hardware circuits across all security levels, dramatically improving implementation security and efficiency.

5. CRYSTALS-Kyber (ML-KEM): Next-Generation Key Encapsulation Mechanism

CRYSTALS-Kyber, officially standardized as FIPS 203 (ML-KEM), is a Key Encapsulation Mechanism (KEM) based on the aforementioned Module-LWE problem. It will become the de facto global standard for securely sharing session keys in TLS 1.3, SSH, and the like in the future.

5.1. Architecture of KEM (Key Encapsulation Mechanism)

In the PQC era, instead of a direct approach like RSA where “the client creates a common key, encrypts it with the server’s public key, and sends it,” a KEM encapsulation framework becomes the standard.

sequenceDiagram participant Client as "Client (Alice)" participant Server as "Server (Bob)" Note over Client: "ML-KEM KeyGen()" Client->>Client: "Generate secret key (sk) and public key (pk)" Client->>Server: "Send public key (pk)" Note over Server: "ML-KEM Encaps()" Server->>Server: "Generate a random shared key (K)" Server->>Server: "Encapsulate K with pk to create ciphertext (c)" Server->>Client: "Send ciphertext (c)" Note over Client: "ML-KEM Decaps()" Client->>Client: "Decrypt ciphertext (c) using secret key (sk)" Client->>Client: "Decapsulate and extract the shared key (K)" Note over Client, Server: "Start encrypted communication (e.g. AES) using the shared key (K)"

5.2. Kyber’s Internal Algorithm Mechanism and the Fujisaki-Okamoto Transform

Kyber’s design is highly sophisticated. First, it constructs a public-key encryption scheme (Kyber.CPAPKE) that is secure only against CPA (Chosen Plaintext Attack), and then adopts a design that upgrades it into a complete KEM that is secure against CCA (Adaptive Chosen Ciphertext Attack) by applying a cryptographically extremely powerful method called the Fujisaki-Okamoto Transform.

The core encryption and decryption mechanisms of CPAPKE are as follows:

Key Generation:
- From a random seed value, generate a matrix $A \in R_q^{k \times k}$ in the NTT domain. The modulus $q$ used is $3329$.
- Sample a secret vector $\vec{s}$ and an error vector $\vec{e}$ with small coefficients from a Centered Binomial Distribution (CBD).
- Calculate $\vec{t} = A\vec{s} + \vec{e}$. The public key is $(A, \vec{t})$, and the secret key is $\vec{s}$. (In reality, $A$ is published as a seed value to save bandwidth).
Encryption:
- Encode the 32-byte message to be shared (shared key material) $m$ into a polynomial.
- Generate a new random vector $\vec{r}$ and small errors $\vec{e_1}, e_2$.
- $\vec{u} = A^T\vec{r} + \vec{e_1}$
- $v = \vec{t}^T\vec{r} + e_2 + \lfloor q/2 \rceil \cdot m$
- The ciphertext is $(\vec{u}, v)$.
Decryption:
- The receiver calculates $v - \vec{s}^T\vec{u}$.
- Expanding this formula mathematically yields the following: $v - \vec{s}^T\vec{u} = (\vec{t}^T\vec{r} + e_2 + \lfloor q/2 \rceil \cdot m) - \vec{s}^T(A^T\vec{r} + \vec{e_1})$
- Substituting $\vec{t} = A\vec{s} + \vec{e}$ here cancels out the main term $\vec{s}^TA^T\vec{r}$.
- What remains is $\lfloor q/2 \rceil \cdot m + (\vec{e}^T\vec{r} + e_2 - \vec{s}^T\vec{e_1})$.
- Since the terms in parentheses are “products and sums of small errors,” they remain sufficiently small values (noise) as a whole. Therefore, by making a threshold judgment on whether each coefficient is close to $0$ or close to $q/2$, the bits (0 or 1) of the original message $m$ can be completely restored without error.

Kyber’s greatest strengths are its overwhelming processing speed and moderate key size. For Kyber768, the public key size is 1,184 bytes and the ciphertext size is 1,088 bytes. While larger compared to RSA-3072 (key size around 384 bytes), it can fit within the MTU (Maximum Transmission Unit) of modern internet communications without packet fragmentation, having almost no adverse effect on network latency.

6. CRYSTALS-Dilithium (ML-DSA): General-Purpose Lattice-Based Digital Signature

In the standardization of digital signatures, algorithms with different design philosophies within the same lattice cryptography approach competed against each stands. Among them, CRYSTALS-Dilithium was selected as FIPS 204 (ML-DSA) for general-purpose digital signatures.

6.1. The Fiat-Shamir with Aborts Paradigm

Like Kyber, Dilithium is a digital signature scheme based on the Module-LWE (and Module-SIS problem). The design base uses an extremely important paradigm called “Fiat-Shamir with Aborts”.

The Fiat-Shamir transform itself is a standard method for converting an interactive zero-knowledge proof protocol into a non-interactive digital signature. The prover (signer) generates a commitment $y$, calculates $w = Ay$, passes it through a hash function to obtain a random challenge $c$, and calculates the response $z = y + cs$.

However, simply applying this to lattice cryptography caused a fatal problem (side-channel-like mathematical leakage) where the distribution of the response $z$ was distorted depending on the value of the secret key $s$, gradually leaking information about the secret key $s$ to an attacker observing many signatures.

The Dilithium design team (Lyubashevsky et al.) introduced a method called “Rejection Sampling”, where if the coefficients of the calculated signature $z$ do not fall within a pre-set safe threshold, the entire signature process is aborted and recalculated from the beginning using a new random number $y$.

As a result, the finally output signature $z$ has a completely uniform distribution independent of the secret key, succeeding in completely preventing mathematical information leakage.

6.2. Dilithium’s Advantages and Ease of Implementation

A major design advantage of Dilithium is that it does not use complex “sampling from a Gaussian distribution” or “floating-point arithmetic” at all in the signature generation process. Because it can be implemented using only sampling from a uniform distribution, simple integer modulo arithmetic, NTT, and a hash function (SHAKE), it is easy to implement securely and in constant-time in a wide range of environments, from embedded microcontrollers to cloud servers. This gives it robust resistance against physical side-channel attacks such as timing attacks.

7. FALCON (FN-DSA): Ultimately Compact Lattice Signature

NIST selected FALCON (Fast-Fourier Lattice-based Compact Signatures over NTRU), another lattice-based signature with different characteristics from Dilithium, as a standardization candidate (currently drafting as FN-DSA).

7.1. NTRU Lattices and Gaussian Sampling

FALCON’s greatest feature is that it uses not the LWE problem but the historical NTRU (N-th degree Truncated polynomial Ring Units) lattices that have existed since 1996. Furthermore, it adopts the “Hash-and-Sign” paradigm based on the GPV (Gentry-Peikert-Vaikuntanathan) framework.

In Hash-and-Sign, the hash value of a message is set as a target point in space, and finding the point on the lattice closest to that point (an approximate solution to the closest vector problem) serves as the signature. To do this, it is necessary to sample points according to a discrete Gaussian distribution using a “high-quality short basis” as the secret key.

FALCON dramatically accelerated this heavy computation using a method called “Fast Fourier Orthogonalization (FFO)”.

7.2. Pros and Cons of FALCON

The overwhelming advantage of FALCON is that its signature size and public key size are extremely small (compact). While the signature size of Dilithium3 is about 3,309 bytes, the signature size of FALCON-512 is only about 666 bytes. The public key is also very small at 897 bytes, making it a lifesaver in environments with extremely limited communication bandwidth, IoT devices, or specific network protocols.

However, there is a significant drawback. Because discrete Gaussian sampling, which involves complex floating-point arithmetic (64-bit IEEE 754), is essential during signature generation, constant-time implementation to prevent timing leakage is extremely difficult, and the code becomes huge. For this reason, FALCON is positioned as a powerful specialized algorithm for specific uses, in contrast to the general-purpose Dilithium.

graph LR A["Requirements for Digital Signatures"] --> B{"What is the top priority constraint?"} B -->|"Simplicity of implementation, versatility, ease of constant-time implementation"| C["Dilithium (ML-DSA)"] B -->|"Minimization of communication bandwidth, compactness of data size"| D["FALCON (FN-DSA)"] C --> E["General-purpose TLS certificates, digital signatures for software"] D --> F["Protocols with strict packet size limits, special environments"]

8. SPHINCS+ (SLH-DSA): Hash-Based Signature Boasting the Strongest Security

To prepare for the worst-case scenario (a rare event) where the security of lattice cryptography is broken by a brilliant mathematician’s breakthrough in the future, NIST formulated FIPS 205 (SLH-DSA), namely SPHINCS+, as a standard with a completely different approach from lattice cryptography.

SPHINCS+ is classified as a hash-based signature. The basis of its security relies solely on “the cryptographic hash functions used (such as SHA-2 and SHAKE256) having collision resistance and one-wayness.” Because it does not depend on mathematical problems with specific algebraic structures like LWE or integer factorization, it boasts extremely robust security (the most conservative security) where even if any powerful quantum algorithm appears in the future, one can simply counter it by increasing the output length of the hash function.

8.1. Stateless Architecture with WOTS+ and FORS

The history of hash-based signatures is old, dating back to Lamport signatures and Winternitz One-Time Signatures (WOTS) in the 1970s. These were disposable keys that could “securely sign only once.” To make them usable multiple times, algorithms like XMSS (eXtended Merkle Signature Scheme) and LMS were developed, combining a Merkle Tree to manage countless one-time keys with a single root hash.

However, XMSS and LMS had a fatal flaw of being “stateful”. It was necessary to strictly record the index state of “which one-time key was used” in non-volatile memory every time a signature was made, and if the state rolled back due to something like restoring a virtual machine snapshot and the same one-time key was used twice, the secret key would leak immediately, and the system would collapse.

SPHINCS+ is a “stateless” hash-based signature that solves this state management hassle. Its core technology is the following combination:

WOTS+ (Winternitz One-Time Signature Plus): A basic one-time signature.
FORS (Forest of Random Subsets): A Few-Time Signature technology. It remains secure even if the same key is reused a few times.
Hyper-Tree: A massive structure of multi-layered Merkle Trees.

When signing with SPHINCS+, instead of managing state, it uses a pseudorandom number to randomly select one of a vast number of FORS keys at the bottom of the Hyper-Tree to sign. Because the number of leaves in the tree is astronomically large, the probability of accidentally picking the same key twice (collision) is negligibly small, resulting in a stateless realization.

The sole and greatest weakness of SPHINCS+ is that its signature size is extremely large. Depending on the parameters, the signature size can reach 17 to 49 kilobytes, and the signature generation speed is also overwhelmingly slower than lattice cryptography. Therefore, rather than for daily web browsing, it is intended for uses where signatures are not made frequently and long-term absolute security is strongly required, such as software update signatures and root Certificate Authority (CA) certificates.

9. Code-Based Cryptography: The Good Old Giant, Classic McEliece

In the NIST standardization process, an important approach still being evaluated as a final candidate for Round 4 is Classic McEliece of code-based cryptography.

Proposed by Robert McEliece in 1978, this algorithm is one of the oldest in the history of public-key cryptography, alongside RSA. It utilizes algebraic geometry codes called “Goppa codes,” where a message is intentionally encrypted with an error (noise vector) added, and only the person holding the parity check matrix of the Goppa code as a secret key can remove the error using powerful error-correcting capabilities to decrypt the original message. This is based on the “Syndrome Decoding Problem”.

$$ \vec{c} = \vec{m} G + \vec{e} $$

(where $G$ is the scrambled generator matrix which is the public key, and $\vec{e}$ is the error vector of weight $t$)

The amazing thing about Classic McEliece is its overwhelming track record: despite more than 40 years passing since its proposal and being exposed to intense cryptanalysis research by cryptographers worldwide, no fundamental vulnerability has ever been discovered. It possesses the most “time-proven robust security” among PQC.

Furthermore, it has the advantage of a very small ciphertext size (only about 100 to 200 bytes). However, it has a fatal flaw in that the public key size is in the megabyte (MB) range. Even at the lowest security level (AES-128 equivalent), the public key is about 250KB, and it exceeds 1MB at higher levels.

For this reason, it cannot be applied at all to uses where the public key is transmitted over a network during every communication, such as in TLS handshakes. However, in special use cases where public keys can be pre-deployed in systems, such as sharing pre-shared keys for VPNs, hardcoding public keys in firmware, or satellite communications, it continues to be considered a highly promising option due to its robust security.

10. Performance Comparison and Trade-offs of Each PQC Algorithm

The performance characteristics of the major algorithms explained so far at typical security levels (equivalent to NIST Level 2-3, AES-128-192 levels) are summarized in the table below.

Algorithm (Standard Name)	Category	Mathematical Basis	Public Key Size	Secret Key Size	Ciphertext/Signature Size	Processing Speed Trend	Main Features and Uses
Kyber768 (ML-KEM)	KEM	Module-LWE	1,184 Bytes	2,400 Bytes	1,088 Bytes	Very fast	Best balance of key size and speed. General-purpose KEM standard such as TLS 1.3.
Dilithium3 (ML-DSA)	Signature	Module-LWE	1,952 Bytes	4,032 Bytes	3,309 Bytes	Fast for both generation and verification	Simple implementation. General-purpose digital signature standard.
FALCON-512 (FN-DSA)	Signature	NTRU Lattice	897 Bytes	1,281 Bytes	666 Bytes	Signature generation is slower, verification is ultra-fast	Minimal signature size. However, requires floating-point operations. For embedded/IoT.
SPHINCS+ (SLH-DSA)	Signature	Hash Function	32 Bytes	64 Bytes	Approx. 17,000 Bytes	Generation is very slow	Mathematical failure risk is almost zero. High security uses like root certificates.
Classic McEliece	KEM	Goppa Code	Approx. 1.04 MB	13,568 Bytes	188 Bytes	Encapsulation is fast	40 years of security track record. Giant public key. For hardcodable environments.

Understanding the Trade-offs

In the world of PQC, there is no single magical algorithm that has “small size, fast speed, and perfect mathematical guarantees.”

The Internet Standard (Kyber / Dilithium): The best balance of performance, making it the most suitable for a drop-in replacement of current RSA/ECC.
Ultimate Conservatism (SPHINCS+): Chosen when one wants absolute insurance against future mathematical breakthroughs, even at the expense of data size or processing speed.
For Special Environments (FALCON / Classic McEliece): Specialized weapons chosen according to environmental constraints, such as when communication bandwidth is extremely narrow or when pre-distribution is possible.

11. Challenges Toward Practical Application and the Practical Solution of “Hybrid Cryptography”

With the completion of standardization by NIST and the official issuance of FIPS standards, the PQC migration of IT infrastructure worldwide has begun in earnest. Google’s Chrome browser, Apple’s iMessage (PQ3 protocol), and network providers like Cloudflare have already implemented PQC support in their protocols and started actual operations.

However, completely switching to new cryptographic algorithms all at once comes with a very high risk. If a brilliant mathematician were to discover a fatal attack method (a mathematical flaw solvable even by classical computers) against lattice cryptography like Kyber a few years from now, entire systems relying on it would instantly become completely defenseless.

A practical and recommended approach to mitigate this uncertainty risk is “Hybrid Cryptography”.

In hybrid cryptography, key exchange is performed using both a classical cryptographic algorithm with a long track record (e.g., Elliptic Curve Cryptography like X25519) and a new PQC algorithm (e.g., Kyber768) simultaneously. Shared key components are generated individually with each algorithm, and finally, a secure Key Derivation Function (KDF) is used to mix the two components to generate the final master secret.

graph TD A["Client"] -->|1. Send X25519 Public Key + Kyber Public Key| B["Server"] B -->|2. Return X25519 Shared Key + Kyber Encapsulated Ciphertext| A A --> C{"Derive Master Secret (KDF)"} B --> C C -->|Input: (X25519 Shared Key) || (Kyber Shared Key)| D["Secure Communication Key (AES-256 / ChaCha20)"] D -->|"Resistant to both quantum threats & classical vulnerabilities"| E["Secure Hybrid Encrypted Communication (TLS 1.3)"]

This achieves a robust two-tiered security: “even if a quantum computer becomes a reality and ECC is broken, Kyber protects the communication,” and conversely, “even if an unknown mathematical flaw is found in Kyber, ECC protects the communication.” A representative example is the X25519MLKEM768 (formerly X25519Kyber768) draft being standardized by the IETF, and communications between current web browsers and cutting-edge servers are already being carried out precisely using this hybrid method.

Furthermore, the concept of Crypto Agility, building a system architecture that “does not overly rely on a specific cryptographic algorithm and can quickly switch to another algorithm (e.g., from Kyber to McEliece, or Dilithium to SPHINCS+) in the event an algorithm fails,” will be an essential requirement in future system development.

12. Conclusion: A New Horizon for Cryptographic Technology

Ironically, quantum computers, the dream technology of humanity, have become the greatest threat to breaking the mathematical defenses of “integer factorization” and “discrete logarithm problems” that we have trusted for many years. However, cryptographers around the world did not succumb to this; they pioneered more complex and profound multi-dimensional mathematical fields such as lattice theory, hash function trees, and error-correcting codes, and built a new defense called Post-Quantum Cryptography (PQC).

The completion of standardizations by NIST for FIPS 203 (ML-KEM), FIPS 204 (ML-DSA), and FIPS 205 (SLH-DSA) is not the goal. It is just the first step in the grand journey of PQC migration that will continue for decades to come. For software engineers and system architects, how to optimally adapt the “increased key sizes” and “changed computational costs” brought by these new algorithms into network protocols and systems will be a major technical challenge moving forward.

The battle between quantum computers and cryptography is an exciting area where humanity’s mathematical exploration and the evolution of technology intersect most fiercely. Through this article, we hope you have deeply understood the beautiful mathematical theories behind PQC and the amazing mechanisms of each algorithm that will shape the future of cybersecurity.

References:

NIST Post-Quantum Cryptography Standardization Program
FIPS 203: Module-Lattice-Based Key-Encapsulation Mechanism Standard
FIPS 204: Module-Lattice-Based Digital Signature Standard
FIPS 205: Stateless Hash-Based Digital Signature Standard

The Day Quantum Computers Become Practical: The Current State in 2026

Fri, 11 Sep 2026 06:00:00 +0900

1. Introduction: Where Quantum Computing Stands in 2026

As of 2026, quantum computing has made a decisive shift from a “theoretical dream” to an “engineering reality.” As the limitations of NISQ (Noisy Intermediate-Scale Quantum) devices—which were mainstream until a few years ago—became clear, research institutions and tech giants around the world shifted their focus toward realizing “FTQC (Fault-Tolerant Quantum Computing).”

In this article, we will delve deep into the current state of quantum computers, incorporating the latest breakthroughs of 2026. In particular, we will detail quantum error correction (surface codes), the difference between physical and logical qubits, advancements in topological quantum computing, and the frontlines of superconducting and ion-trap architectures.

2. Fundamentals of Quantum States and Fidelity

The qubit, the fundamental unit of a quantum computer, differs from a classical bit (0 or 1) in that it can exist in a superposition of 0 and 1. The state of a single qubit is represented as a vector on a Hilbert space as follows:

$$ |\psi\rangle = \alpha|0\rangle + \beta|1\rangle $$

Here, $\alpha$ and $\beta$ are complex probability amplitudes that satisfy the following normalization condition:

$$ |\alpha|^2 + |\beta|^2 = 1 $$

An extremely important metric for measuring the performance of quantum computation is Fidelity. The fidelity $F$ between an ideal quantum state $|\psi\rangle$ and an actual density matrix $\rho$ that has degraded into a mixed state due to noise is defined as follows:

$$ F(\rho, |\psi\rangle) = \langle \psi | \rho | \psi \rangle $$

As of 2026, the fidelity of 2-qubit gates (e.g., CNOT and CZ gates) has stably surpassed the 99.99% barrier (the so-called “four nines”) in superconducting architectures. This significantly exceeds the threshold for error correction using surface codes (about 99%), making it one of the biggest breakthroughs toward practical application.

3. The Limits of the NISQ Era and the Paradigm Shift to FTQC

The late 2010s to the early 2020s was the era of NISQ (Noisy Intermediate-Scale Quantum)—devices with tens to hundreds of qubits without error correction. However, NISQ had clear limitations.

As the circuit depth increases, errors accumulate exponentially, making it impossible to obtain meaningful computation results. The overall success probability $P_{success}$ at a circuit depth $D$ decays with respect to the single-gate fidelity $f$ and the number of gates $N$ as follows:

$$ P_{success} \approx f^N $$

If $f = 0.99$ and 1000 gates are applied, $0.99^{1000} \approx 4.3 \times 10^{-5}$, and the result is almost entirely buried in random noise. For this reason, in 2026, resources are heavily concentrated on generating Logical Qubits rather than directly scaling up NISQ algorithms (like VQE and QAOA).

4. Quantum Error Correction and Logical Qubits: The Forefront of Surface Codes

Quantum Error Correction (QEC) is a technology that encodes multiple “physical qubits” to create a single “logical qubit,” detecting and correcting errors. The most promising approach currently is the Surface Code.

4.1 Structure of the Surface Code

In a surface code, qubits are arranged in a 2-dimensional grid. Data qubits (which hold the actual information) and measurement qubits (for syndrome measurement) are arranged in a checkerboard pattern.

graph TD A["Data Qubit (D1)"] --- B["Measure Qubit (M1)"] B --- C["Data Qubit (D2)"] C --- D["Measure Qubit (M2)"] D --- E["Data Qubit (D3)"] B --- F["Data Qubit (D4)"] D --- G["Data Qubit (D5)"] style A fill:#e1f5fe,stroke:#039be5 style C fill:#e1f5fe,stroke:#039be5 style E fill:#e1f5fe,stroke:#039be5 style F fill:#e1f5fe,stroke:#039be5 style G fill:#e1f5fe,stroke:#039be5 style B fill:#fff3e0,stroke:#fb8c00 style D fill:#fff3e0,stroke:#fb8c00

Bit-flip (X errors) and phase-flip (Z errors) are constantly monitored using the stabilizer operators $S_x$ and $S_z$.

$$ S_x = \prod_{i \in \text{star}} X_i, \quad S_z = \prod_{j \in \text{plaquette}} Z_j $$

A significant advancement in 2026 is that the “Break-even point” has been completely surpassed. In other words, the noise removed by error correction has become greater than the noise introduced by the extra circuitry required for it, allowing the lifespan of a logical qubit to exceed that of a physical qubit by orders of magnitude.

4.2 The Quantum Error Correction Cycle

Error correction functions as a continuous feedback loop.

sequenceDiagram participant D as "Data Qubits" participant M as "Ancilla/Measure Qubits" participant C as "Classical Controller" loop "Syndrome Extraction Cycle (approx 1 microsec)" D->>M: "Entangle (CNOT/CZ)" M->>C: "Measure State (Syndrome)" C->>C: "Decode Syndrome (e.g. Minimum Weight Perfect Matching)" C-->>D: "Apply Pauli Correction (if necessary)" end

Currently, the technology to execute this classical decoding process (syndrome analysis) in nanoseconds using FPGAs or dedicated ASICs has been established, and real-time error correction has entered the practical stage.

5. Evolution of Hardware Architectures (2026 Edition)

Quantum hardware in 2026 is evolving primarily along three axes: “Superconducting,” “Ion-Trap,” and “Topological.”

5.1 Integration of Superconducting Qubits

The superconducting approach is a field led by companies like IBM and Google, with Transmon qubits using Josephson junctions being the mainstream. In 2026, megachips integrating thousands to ten thousand physical qubits on a single chip became a reality.

Notably, Quantum Interconnects (module-to-module quantum communication) have been established. Quantum teleportation between chips using microwave photons has been implemented at a commercial level, making it possible to bypass the size limitations of a single dilution refrigerator.

5.2 2D Scaling and Optical Interconnects for Ion Traps

The ion-trap architecture (led by Quantinuum, IonQ, etc.) uses the internal energy states of ions suspended in a vacuum as qubits. Compared to superconducting methods, they boast extremely long T1/T2 coherence times and have the advantage of all-to-all connectivity.

The 2026 breakthrough involved expanding the QCCD (Quantum Charge Coupled Device) architecture into two dimensions and generating high-speed entanglement between multiple traps using photonic interconnects. This drastically improved the slow gate speeds and scalability issues that were weaknesses of the ion-trap method.

5.3 Topological Quantum Computing: Controlling Anyons

Topological quantum computing, long considered theoretical, has finally entered the phase of experimental demonstration in 2026. This approach, promoted by Microsoft and others, uses non-Abelian anyons called “Majorana Zero Modes.”

Quantum gates are executed through an operation called “Braiding,” which involves swapping the positions of anyon particles.

$$ |\psi_{final}\rangle = B_{ij} |\psi_{initial}\rangle $$

Here, $B_{ij}$ is the braiding operator. Because the topological approach relies on the global topology of the “knots” rather than the local state of particles to store information, it is inherently robust against environmental noise (hardware-level fault tolerance). In 2026, the world’s first generation of high-fidelity topological logical qubits was confirmed, drawing attention as a powerful shortcut to FTQC.

6. Roadmap to Practical Application and Future Outlook

To truly demonstrate Quantum Advantage, where quantum computers overwhelm classical computers (supercomputers) in fields like “chemical computation,” “materials science,” and “financial modeling,” thousands of logical qubits are required.

gantt title "Quantum Computing Roadmap (Revised 2026)" dateFormat YYYY axisFormat %Y section "NISQ Era" "Noisy Qubits (<1000)" :done, 2018, 2024 section "Early FTQC" "Break-even Point Demonstration" :done, 2024, 2026 "Hundreds of Logical Qubits" :active, 2026, 2028 section "Full-Scale FTQC" "1000+ Logical Qubits (Commercial App)" : 2028, 2030 "Universal Fault-Tolerant Quantum Computer" : 2030, 2035

6.1 Current Challenges and the Future

The biggest challenges as of 2026 are the cooling capacity of the massive cryostats (dilution refrigerators) needed to maintain ultra-low temperatures, and the wiring (I/O bottleneck) connecting room-temperature control equipment to the cryogenic quantum chips. In response, the development of Cryo-CMOS controller chips that operate in cryogenic environments is advancing rapidly.

Conclusion

2026 will likely be recorded in the history of quantum computing as “the first year of logical qubit scaling.” With the demonstration of error correction algorithms, the modularization of hardware, and rapid progress in the topological approach, “the day they become practical” is no longer a tale of the distant future but a concrete milestone to look forward to within the next few years. For developers and companies in the quantum algorithm space, now is the time to seriously invest in quantum-native problem solving.

This article was written based on the latest quantum computing research papers and industry trends as of 2026.

How to Solve the Traveling Salesperson Problem (TSP) in Mathematica

Wed, 12 Oct 2022 19:05:58 +0900

Solving the Traveling Salesperson Problem with Mathematica

Problem

電車でこんな広告を見かけました😁📸 pic.twitter.com/iXEgvtXrpL
— 早稲田大学早水桃子研究室 (@hayamizu_lab) October 11, 2022

Solution

1

d=SparseArray[{{1,2}->10,{2,1}->10,{1,5}->15,{5,1}->15,{1,4}->12,{4,1}->12,{1,3}->20,{3,1}->20,{2,5}->10,{5,2}->10,{3,4}->10,{4,3}->10,{3,8}->30,{8,3}->30,{3,7}->20,{7,3}->20,{3,6}->25,{6,3}->25,{4,5}->15,{5,4}->15,{4,8}->20,{8,4}->20,{5,9}->18,{9,5}->18,{5,8}->15,{8,5}->15,{6,7}->5,{7,6}->5,{7,8}->35,{8,7}->35,{8,9}->12,{9,8}->12},{9,9},Infinity];

We create a matrix using the SparseArray function. Each element represents the distance between cities at the row and column of that element. For example, the first element {1,2}->10 means the distance between 1 and 2 is 10. The second to last element {9,9} indicates the size of the matrix, and the final element Infinity means the length of paths between unspecified cities is infinite, meaning there is no path.

1

{len,tour}=FindShortestTour[{1,2,3,4,5,6,7,8,9},DistanceFunction->(d[[#1,#2]]&)]

You can easily solve the traveling salesperson problem with the FindShortestTour function. {1,2,3,4,5,6,7,8,9} represents the city numbers. DistanceFunction->(d[[#1,#2]]&) passes the matrix d which represents the distance between cities.

Output

1

{137, {1, 2, 5, 9, 8, 7, 6, 3, 4}}

The output gives the shortest distance and the tour route for it. The shortest distance is 137, and the route is 1→2→5→9→8→7→6→3→4→1. Converting this to ABC order gives A, B, E, I, H, G, F, C, D.