Math on kenji.blog

【For Beginners】 Deciphering the Mathematical Structure of the Transformer Model

Fri, 11 Sep 2026 22:00:00 +0900

Introduction: Why Learn the Mathematics of Transformers?

It is no exaggeration to say that the “Transformer” is the architecture that rewrote the history of modern Natural Language Processing (NLP) and AI as a whole. First proposed in the 2017 paper “Attention Is All You Need” by Google researchers, this model serves as the heart of Large Language Models (LLMs) that are currently taking the world by storm, such as OpenAI’s GPT series (the foundational technology of ChatGPT), Google’s BERT, and Anthropic’s Claude.

However, while qualitative explanations like “understanding context using Attention mechanisms” are commonly seen regarding how Transformers work, surprisingly few resources dive deep into the mathematical structure behind it for beginners. To truly understand how AI processes “words” as “mathematical formulas” and generates incredibly natural sentences, deciphering its mathematical mechanisms is essential.

This article is aimed at those with a basic understanding of mathematics and programming (those who grasp high school-level concepts of matrices and derivatives). It thoroughly and clearly uncovers the mathematical structures of the Transformer’s core components: the “Self-Attention mechanism,” the “Query-Key-Value (Q/K/V) model,” “normalization using the Softmax function,” and “Positional Encoding.”

You might be overwhelmed by the list of mathematical formulas, but each calculation has a clear “meaning.” By the time you finish reading this article, you should understand that the Transformer is not just a magical black box, but an exquisitely designed crystallization of mathematics and statistics.

1. Limitations of Conventional Methods and the Innovativeness of the Transformer

Before the advent of the Transformer, the mainstream of natural language processing was Recurrent Neural Networks (RNNs) and their derivative, LSTM (Long Short-Term Memory). RNNs are designed to process time-series data, reading sentences sequentially from the beginning, word by word.

However, RNNs had two fatal weaknesses:

Difficulty in learning long-term dependencies: As sentences get longer, the information of the words inputted at the beginning fades by the time it reaches the end (the vanishing gradient problem).
Inability to compute in parallel: Because words must be processed sequentially, large-scale parallel computation using GPUs is difficult, requiring an enormous amount of time for training.

The Transformer caused a paradigm shift by completely discarding the RNN structure and grasping context using only “Attention.” This allowed for no loss of information no matter how long the sequence is, and made it possible to maximize GPU performance by parallelizing computations.

2. Overall Architecture of the Transformer

First, let’s take a bird’s-eye view of the overall Transformer architecture. The Transformer is broadly composed of two blocks: the “Encoder” and the “Decoder.” Taking a translation task as an example, the Encoder converts the input language (e.g., English) into mathematical vector representations, and the Decoder generates the output language (e.g., Japanese) based on those vector representations.

The following diagram is a simplified internal structure of the Encoder block.

graph TD A["Input Tokens"] --> B["Input Embedding"] B --> C["Positional Encoding"] C --> D["Multi-Head Self-Attention"] D --> E["Add & Layer Normalization"] E --> F["Feed Forward Network"] F --> G["Add & Layer Normalization"] G --> H["Output to Next Layer"] C -.->|"Residual Connection"| E E -.->|"Residual Connection"| G

From here, let’s look step by step at the mathematical operations being performed in each component.

3. Word Vectorization and Positional Encoding

Computers cannot understand text as it is. The inputted text is first divided into units called “Tokens,” and each is converted into a fixed-length vector. This is Input Embedding.

3.1 Mathematics of Input Embedding

Let the size of the vocabulary be $V$, and the dimensionality of the embedding vector be $d_{model}$ (in the original paper, $d_{model} = 512$). Each word $w_i$ is converted into a vector $x_i \in \mathbb{R}^{d_{model}}$ using the embedding matrix $W_E \in \mathbb{R}^{V \times d_{model}}$.

$$ x_i = W_E \cdot \text{one\_hot}(w_i) $$

As a result, the entire sentence is represented as a matrix $X \in \mathbb{R}^{N \times d_{model}}$ (where $N$ is the length of the sentence).

3.2 The Need for Positional Encoding and its Formulas

Unlike RNNs, the Transformer does not process words sequentially but processes all words in parallel simultaneously. This is a significant advantage in terms of computational speed, but at the same time, it causes the problem that important information of “word order” is lost. For example, “A dog bites a man” and “A man bites a dog” have the exact same set of input words, but their meanings are completely different.

Positional Encoding was devised to provide this word order information to the model. The Positional Encoding $PE$ for the $i$-th dimension of a word at position $pos$ is calculated using the following trigonometric functions:

$$ PE_{(pos, 2i)} = \sin\left(\frac{pos}{10000^{2i/d_{model}}}\right) $$ $$ PE_{(pos, 2i+1)} = \cos\left(\frac{pos}{10000^{2i/d_{model}}}\right) $$

Here, $pos$ is the position of the word ($0, 1, 2, \dots, N-1$), and $i$ is the index of the vector’s dimension ($0, 1, \dots, d_{model}/2 - 1$).

Why Use Sine and Cosine?

At first glance, it looks like a very complex and strange mathematical formula, but there is a profound mathematical reason for this. By using trigonometric functions, the model can easily learn not only the “absolute position” but also the difference in “relative position”.

Recall the addition theorems of trigonometric functions learned in high school math:

$$ \sin(\alpha + \beta) = \sin\alpha \cos\beta + \cos\alpha \sin\beta $$ $$ \cos(\alpha + \beta) = \cos\alpha \cos\beta - \sin\alpha \sin\beta $$

The Positional Encoding of a position $pos + k$, which is offset by $k$ from a certain position $pos$, can be expressed as a linear combination of the Positional Encoding of position $pos$. In other words, using a matrix $M_k$, it can be written as follows:

$$ PE_{pos+k} = M_k \cdot PE_{pos} $$

This allows the Attention mechanism to easily recognize the relative distance, or “how far apart” words are, through dot product calculations. Another advantage is that by combining multiple sine and cosine waves of different wavelengths, a unique position vector can be generated no matter how long the sentence is.

The final input matrix $X_{input}$ is the sum of the word embedding vectors and this positional encoding.

$$ X_{input} = X + PE $$

4. The Profound Mathematics of Self-Attention

We will finally step into the Self-Attention mechanism, the most critical component of the Transformer. The purpose of Self-Attention is “to calculate the degree of relevance between all words in a sentence and update the vector of each word into a richer representation that takes context into account.”

Here, an analogy of a “search system” is used.

Query (Q): The query (search term). “What information am I looking for right now?”
Key (K): The key (heading). “What information do I have?”
Value (V): The value (entity). “What information do I actually provide?”

4.1 Generation of Matrices $Q, K, V$

For the input matrix $X \in \mathbb{R}^{N \times d_{model}}$ (we ignore batch size here for simplicity), we calculate the query $Q$, key $K$, and value $V$ by multiplying it with learnable weight matrices $W^Q, W^K, W^V \in \mathbb{R}^{d_{model} \times d_k}$. (Usually $d_k = d_v = d_{model} / h$)

$$ Q = X W^Q $$ $$ K = X W^K $$ $$ V = X W^V $$

Here, $Q, K, V$ are all matrices in $\mathbb{R}^{N \times d_k}$.

4.2 Calculation of Attention Scores (Dot Product)

To measure how much each word’s Query is related to the Keys of all other words, we calculate the dot product of the vectors. Written as a matrix operation, it looks like this:

$$ \text{Scores} = Q K^T $$

Each element $s_{ij}$ of the matrix $\text{Scores} \in \mathbb{R}^{N \times N}$ obtained by this calculation represents the dot product of the $i$-th word’s Query and the $j$-th word’s Key, that is, the “strength of relevance”.

4.3 Scaling (Scale)

There is one problem with calculating scores via dot products. As the dimensionality $d_k$ of the vectors becomes larger, the values of the dot product can become extremely large or small.

Let’s prove this mathematically. Assume that each element $q$ of the query and $k$ of the key follows an independent standard normal distribution: $q \sim \mathcal{N}(0, 1)$ and $k \sim \mathcal{N}(0, 1)$. We find the mean and variance of the dot product $q \cdot k = \sum_{i=1}^{d_k} q_i k_i$. Mean: $\mathbb{E}[q_i k_i] = \mathbb{E}[q_i] \mathbb{E}[k_i] = 0 \times 0 = 0$, so the mean of the sum is also $0$. Variance: The variance of $q_i k_i$ is, from independence, $\text{Var}(q_i k_i) = \mathbb{E}[(q_i k_i)^2] - (\mathbb{E}[q_i k_i])^2 = 1 \times 1 - 0 = 1$. Therefore, the variance of the entire dot product is equal to the number of dimensions $d_k$.

$$ \text{Var}(q \cdot k) = d_k $$

When the variance becomes large, in the Softmax function applied subsequently, the gradients for values other than the maximum become extremely small, leading to “vanishing gradients”, and learning stops progressing. To prevent this, the scores are divided (scaled) by $\sqrt{d_k}$ so that the variance is constantly kept at $1$.

$$ \text{Scaled Scores} = \frac{Q K^T}{\sqrt{d_k}} $$

4.4 Probabilization by Softmax Function

To convert the obtained scores into a probability distribution (weights) that sums to $1$, the Softmax function is applied row by row.

$$ a_{ij} = \text{softmax}(s_i)_j = \frac{\exp(s_{ij} / \sqrt{d_k})}{\sum_{m=1}^N \exp(s_{im} / \sqrt{d_k})} $$

The matrix $A \in \mathbb{R}^{N \times N}$ is called the Attention Weight matrix. Looking at each row $i$ of this matrix, it expresses “how much attention should be paid to other words $j$ in order to understand word $i$” as a value between 0 and 1.

4.5 Weighted Sum of Value

Finally, using the obtained Attention Weight matrix $A$, we calculate the weighted sum of the Value matrix $V$.

$$ \text{Output} = A V = \text{softmax}\left(\frac{Q K^T}{\sqrt{d_k}}\right) V $$

The matrix $Z \in \mathbb{R}^{N \times d_v}$ output by this operation is a collection of “word vector representations updated to account for context”. This is the complete picture of the Scaled Dot-Product Attention defined in the paper.

5. Multi-Head Attention

A single Attention calculation (single-head) might only capture context from one perspective (for example, “grammatical relationships”). Therefore, to simultaneously capture the diverse semantic and syntactic relationships of language (such as “subject and predicate” or “pronouns and their referents”), Multi-Head Attention was introduced.

The generation of $Q, K, V$ and Attention calculation described earlier are performed in parallel $h$ times (the number of heads. In the original paper, $h=8$).

$$ \text{head}_i = \text{Attention}(X W_i^Q, X W_i^K, X W_i^V) $$

Here, $W_i^Q, W_i^K, W_i^V \in \mathbb{R}^{d_{model} \times d_k}$ are learnable weight matrices dedicated to the $i$-th head.

The results output from each head, $\text{head}_i \in \mathbb{R}^{N \times d_v}$, are concatenated horizontally.

$$ \text{Concat}(\text{head}_1, \dots, \text{head}_h) \in \mathbb{R}^{N \times (h \cdot d_v)} $$

Usually, it is set such that $h \cdot d_v = d_{model}$, so the concatenated dimension returns to the original $d_{model}$. Finally, this matrix is multiplied by a weight matrix $W^O \in \mathbb{R}^{d_{model} \times d_{model}}$ to obtain the final output.

$$ \text{MultiHead}(Q, K, V) = \text{Concat}(\text{head}_1, \dots, \text{head}_h) W^O $$

graph TD X["Input X"] --> Q1["Q1"] X --> K1["K1"] X --> V1["V1"] Q1 & K1 & V1 --> H1["Head 1"] X --> Q2["Q2"] X --> K2["K2"] X --> V2["V2"] Q2 & K2 & V2 --> H2["Head 2"] X --> QN["..."] X --> KN["..."] X --> VN["..."] QN & KN & VN --> HN["Head h"] H1 & H2 & HN --> C["Concatenate"] C --> WO["Multiply by WO"] WO --> OUT["Multi-Head Output"]

6. Feed-Forward Neural Network (FFN)

The output of Multi-Head Attention is next inputted into the Position-wise Feed-Forward Network (FFN). This is a two-layer fully connected neural network applied “independently to each position (word)” in the sequence.

Expressed as a formula, it is as follows:

$$ \text{FFN}(x) = \max(0, x W_1 + b_1) W_2 + b_2 $$

Here, $\max(0, z)$ represents the ReLU (Rectified Linear Unit) activation function (recently, GELU or SwiGLU are also often used in modern models).

The role of this network is extremely important. While the Attention mechanism learns “relationships between words (spatial/sequential relationships)”, the FFN is responsible for “non-linear feature transformation of each word vector itself”. Usually, the dimension is temporarily expanded greatly by the weights of the first layer $W_1$ (for example, expanding by 4 times from $d_{model}=512$ to $d_{ff}=2048$), and after performing complex calculations in the feature space, it is returned to the original dimension by the weights of the second layer $W_2$. Through this “expansion and contraction of dimensions”, the expressive power of the model is dramatically enhanced.

7. Residual Connection and Layer Normalization

In deep learning, as the layers of a network become deeper, problems arise where gradients vanish or explode during training, making it impossible to learn properly. To prevent this, Residual Connections and Layer Normalization are placed around each sublayer (Attention and FFN) of the Transformer.

Written mathematically, the output of the sublayer is processed as follows:

$$ \text{Output} = \text{LayerNorm}(x + \text{Sublayer}(x)) $$

7.1 Residual Connection ($x + \text{Sublayer}(x)$)

The input $x$ is directly added to the output of the sublayer. By doing this, gradients can propagate directly to shallower layers through shortcuts during backpropagation, stabilizing learning even when the layers are deepened.

7.2 Mathematics of Layer Normalization

Layer Normalization is a technique that calculates the mean and variance along the feature dimension direction to normalize data. For an input with batch size $B$, sequence length $N$, and dimensionality $d_{model}$, normalization is performed on a single word vector $x \in \mathbb{R}^{d_{model}}$.

Calculate the mean $\mu$ and variance $\sigma^2$:

$$ \mu = \frac{1}{d_{model}} \sum_{i=1}^{d_{model}} x_i $$ $$ \sigma^2 = \frac{1}{d_{model}} \sum_{i=1}^{d_{model}} (x_i - \mu)^2 $$

Then, obtain the normalized output $\hat{x}$:

$$ \text{LN}(x) = \frac{x - \mu}{\sqrt{\sigma^2 + \epsilon}} \odot \gamma + \beta $$

(where $\epsilon$ is a small constant to prevent division by zero, and $\gamma, \beta$ are learnable scale and shift parameters)

The reason for adopting Layer Normalization instead of Batch Normalization is that when processing sequential data of variable length, like sentences, batch-wise statistics tend to become unstable. Thanks to Layer Normalization, the Transformer is capable of stable learning independent of batch size.

8. Decoder-Specific Structures: Masked Attention and Cross-Attention

The structure explained so far is for the Encoder. In the Decoder block that generates text, the structure is slightly different.

8.1 Masked Multi-Head Attention

The role of the Decoder is “to predict the next word from past words”. Therefore, “looking ahead at future words” during training would be cheating. The mathematical operation to prevent this is Masking.

To the score matrix $Q K^T$, we add a mask matrix $M$ that sets extremely small values close to $-\infty$ for the upper triangular part (corresponding to future information).

$$ M_{ij} = \begin{cases} 0 & (i \le j) \\ -\infty & (i > j) \end{cases} $$ $$ \text{Masked Attention}(Q, K, V) = \text{softmax}\left(\frac{Q K^T + M}{\sqrt{d_k}}\right) V $$

When calculating the Softmax function, since $\exp(-\infty) = 0$, the Attention Weight for future words becomes exactly $0$. This enables autoregressive generation while preserving causality.

8.2 Encoder-Decoder Cross-Attention

The second sublayer of the Decoder is Cross-Attention, which references the output from the Encoder. Here, $Q$ is generated from the previous decoder layer, while $K$ and $V$ are generated from the output of the final encoder layer.

$$ Q_{decoder} = X_{dec} W^Q $$ $$ K_{encoder} = X_{enc} W^K $$ $$ V_{encoder} = X_{enc} W^V $$

Through this calculation, in tasks like translation, the model can learn “which parts of the original foreign language sentence the currently translating word is strongly related to”.

9. Computational Complexity and Mathematics of Modern Optimization

The Transformer is a brilliant model, but it also has “weaknesses” due to its mathematical structure. Consider the computational complexity of Self-Attention. Calculating the score matrix $Q K^T$ involves multiplying an $(N \times d_k)$ matrix with a $(d_k \times N)$ matrix, so its computational complexity is $O(N^2 \cdot d_{model})$.

In other words, the computational complexity and memory usage increase quadratically with respect to the sequence length $N$. This is not a problem when sentences are short, but if you try to input an enormous context like a whole book into an LLM, $N$ reaches tens to hundreds of thousands, and conventional Attention calculations will immediately exhaust GPU memory.

To break this curse of $O(N^2)$, various optimizations from mathematical and hardware approaches have been proposed in recent years. A representative example is FlashAttention. FlashAttention is an algorithm that divides the Attention calculation into tiles (Tiling) to minimize data transfer (memory access) between GPU memory hierarchies (SRAM and HBM). Even though mathematically it outputs exactly the same result as standard Attention (Exact Attention), it achieves dramatic speedups and memory reduction through hardware-level optimization, enabling the realization of long-context models like GPT-4.

In addition, research on Sparse Attention and Linear Attention, which approximate the computational complexity to $O(N \log N)$ or $O(N)$, is also actively being conducted.

10. Implementation Concept (PyTorch-style Pseudocode)

When translating the mathematical structures up to this point into actual programming code (Python / PyTorch), you’ll see that it can be written surprisingly simply. Here is the pseudocode for the core part of Self-Attention.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


import torch
import torch.nn.functional as F
import math

def scaled_dot_product_attention(q, k, v, mask=None):
 # Shape of q, k, v: [batch_size, num_heads, seq_length, d_k]
 d_k = q.size(-1)

 # 1. Score calculation via dot product: Q * K^T
 # Transpose the last two dimensions to calculate matrix multiplication
 scores = torch.matmul(q, k.transpose(-2, -1))

 # 2. Scaling
 scores = scores / math.sqrt(d_k)

 # 3. Masking (for Masked Attention)
 if mask is not None:
 scores = scores.masked_fill(mask == 0, -1e9)

 # 4. Probabilization by Softmax
 attention_weights = F.softmax(scores, dim=-1)

 # 5. Multiplication with Value matrix
 output = torch.matmul(attention_weights, v)

 return output, attention_weights

You can intuitively see that $Q K^T / \sqrt{d_k}$ expressed mathematically is implemented as torch.matmul(q, k.transpose(-2, -1)) / math.sqrt(d_k). The fact that mathematical theories can be realized in just a few lines of code with the help of advanced optimization libraries is a highly fascinating aspect of deep learning.

Conclusion: The Shape of “Intelligence” Seen Through Mathematical Formulas

In this article, we have deciphered the deep mathematical structures of the Transformer model.

Embedding maps words into a multi-dimensional vector space, Positional Encoding represents position information through the composition of triangular waves, and the Self-Attention mechanism is a matrix dot product calculation born from an information retrieval analogy. Each of these components is merely an accumulation of fundamental mathematics such as linear algebra, calculus, and probability statistics.

However, when these simple matrix operations are layered over and over, learning patterns from massive datasets through billions or hundreds of billions of parameters, a “shape of intelligence” emerges—one that seems to understand our “words”, perform logical reasoning, and sometimes generate creative ideas.

As the provocative title “Attention Is All You Need” suggests, the beauty of this architecture, which discards complex recurrent or convolutional processing and specializes purely in calculating “attention (relevance)”, lies exactly in its mathematical simplicity.

While there is a possibility that new architectures surpassing the Transformer (such as Mamba, a State Space Model) may appear in the future, the mathematical framework of “context understanding through Attention” built by the Transformer will surely be etched in the history of AI forever.

If you have the opportunity to use LLMs like ChatGPT or Claude in the future, imagine the trillions of $Q K^T$ matrix multiplications being calculated per second in the background, with the Softmax function spitting out probabilities. Your resolution regarding the technology will increase, and you should find the world of AI even more fascinating.

References

Vaswani, A., et al. (2017). “Attention Is All You Need.” Advances in Neural Information Processing Systems.
Alammar, J. (2018). “The Illustrated Transformer.”

This article was written as a guide for those learning the mathematical foundations of natural language processing and AI. If you have any questions or discussions, please let us know in the comments!

Implementing General Number Field Sieve (GNFS) in C++ | Mechanism of RSA Cryptanalysis Algorithm

Sat, 05 Sep 2026 13:04:59 +0900

[Complete Anatomy] Understanding the Strongest Cryptanalysis Algorithm “GNFS” by Implementing it in C++

The “RSA cryptography” fundamentally supports the modern Internet. Its robustness relies on the mathematical belief that “factoring huge composite numbers is practically impossible with current computers.”

However, humanity has never given up. Currently, for classical computers (regular computers, not quantum computers), there exists the **strongest and most advanced algorithm of humanity ** for performing giant prime factorizations. That is the “General Number Field Sieve (GNFS)”.

In this article, we will strictly model the state-of-the-art computational logic of GNFS in C++ (using the multiple-precision integer boost::multiprecision from the Boost library), publish the entire implementation code, and thoroughly explain the depths of “algebraic number theory” behind it.

Please enjoy the mystery of mathematics and the brute force of computer science that wrestles it down, along with the source code.

1. GNFS State-of-the-Art Logic Framework (Full Source Code)

First, here is the full picture of the C++ implementation of GNFS that we will explain this time. The actual number field sieve (such as CADO-NFS) is an ultra-massive distributed system spanning hundreds of thousands of lines, but this code extracts the “5 essential pipelines (phases)” that make up GNFS, designs them as classes, and models them in a minimal configuration without losing their mathematical meaning.

 1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227


#include <iostream>
#include <vector>
#include <cmath>
#include <map>
#include <set>
#include <chrono>
#include <boost/multiprecision/cpp_int.hpp>

// Using Boost.Multiprecision for multiple-precision integers
using namespace boost::multiprecision;

// ============================================================================
// [SOTA GNFS] General Number Field Sieve State-of-the-Art Logic Framework
//
// This code strictly models the 5 pipelines of state-of-the-art GNFS used in
// CADO-NFS etc., as a class design in C++ (Boost).
// ============================================================================

struct Relation {
 int64_t a;
 int64_t b;
 std::vector<uint32_t> rational_primes;
 std::vector<uint32_t> algebraic_primes;
};

// ============================================================================
// Phase 1: Polynomial Selection (KleinJung's algorithm)
// ============================================================================
class PolynomialSelector {
public:
 int degree;
 std::vector<cpp_int> f; // Algebraic side polynomial f(x)
 std::vector<cpp_int> g; // Rational side polynomial g(x) = x - m
 cpp_int m;

 PolynomialSelector(int d) : degree(d) {}

 // Initial polynomial generation based on base-m expansion (actually uses more advanced lattice basis reduction LLL)
 void select(const cpp_int& N) {
 std::cout << "[Phase 1] Polynomial Selection (Degree " << degree << ") starting..." << std::endl;
 // Simple base-m expansion (degree d)
 // m = N^(1/d)
 cpp_int N_copy = N;
 m = 1;
 // Simple approximation of m (approximation without using Boost functions)
 cpp_int low = 1, high = N;
 while (low <= high) {
 cpp_int mid = low + (high - low) / 2;
 cpp_int p = 1;
 for(int i=0; i<degree; ++i) p *= mid;
 if (p <= N) { m = mid; low = mid + 1; }
 else { high = mid - 1; }
 }

 f.resize(degree + 1);
 cpp_int temp = N;
 for (int i = 0; i <= degree; ++i) {
 f[i] = temp % m;
 temp /= m;
 }

 g = {-m, 1}; // g(x) = x - m

 std::cout << " -> m = " << m << std::endl;
 std::cout << " -> f(x) = ";
 for(int i = degree; i >= 0; --i) {
 std::cout << f[i] << "x^" << i << (i > 0 ? " + " : "");
 }
 std::cout << "\n[Phase 1] Complete." << std::endl;
 }
};

// ============================================================================
// Phase 2: Lattice Sieving
// ============================================================================
// In recent GNFS, instead of Line Sieve, Special-q Lattice Sieving by
// Franke-Kleinjung et al. is the de facto standard.
class LatticeSieve {
 uint32_t rational_bound;
 uint32_t algebraic_bound;
 std::vector<uint32_t> rational_fb;
 std::vector<uint32_t> algebraic_fb;

public:
 LatticeSieve(uint32_t rb, uint32_t ab) : rational_bound(rb), algebraic_bound(ab) {}

 void generate_factor_bases() {
 std::cout << "[Phase 2] Generating Factor Bases (Rational Bound: " << rational_bound << ", Algebraic Bound: " << algebraic_bound << ")" << std::endl;
 // (Omitted) In reality, it generates primes and filters them using Legendre symbols, etc.
 }

 std::vector<Relation> sieve(const PolynomialSelector& poly) {
 std::cout << "[Phase 2] Special-q Lattice Sieving active..." << std::endl;
 std::vector<Relation> relations;
 // Mock implementation: Actual lattice sieving scans hundreds of GB of memory space block by block.
 // It maps (a, b) pairs to lattices for each special prime q (a = i*q + j*...),
 // and executes a sieve that maximizes cache efficiency.

 // Add one dummy relation for demo
 Relation r; r.a = 17; r.b = 3;
 r.rational_primes = {2, 5};
 r.algebraic_primes = {3, 7};
 relations.push_back(r);

 std::cout << "[Phase 2] Found " << relations.size() << " relations." << std::endl;
 return relations;
 }
};

// ============================================================================
// Phase 3: Filtering (Singleton removal and clique merging)
// ============================================================================
class Filter {
public:
 void reduce_matrix(std::vector<Relation>& relations) {
 std::cout << "[Phase 3] Filtering Relations..." << std::endl;
 // 1. Singleton removal (removing relations with primes that appear only once)
 // 2. Clique merging (merging relations to make a sparse matrix denser)
 // In reality, it compresses a matrix of hundreds of millions of rows down to several million using algorithms like Union-Find.
 std::cout << "[Phase 3] Matrix size reduced optimally." << std::endl;
 }
};

// ============================================================================
// Phase 4: Linear Algebra over GF(2) (Block Wiedemann method)
// ============================================================================
class LinearAlgebraGF2 {
public:
 // In modern supercomputing environments, the Block Wiedemann method (Coppersmith implementation),
 // which is more suitable for distributed computing than the Block Lanczos method, is used as the state-of-the-art.
 std::vector<std::vector<int>> solve_nullspace(const std::vector<Relation>& relations) {
 std::cout << "[Phase 4] Block Wiedemann algorithm over GF(2) starting..." << std::endl;
 // Iterates matrix-vector multiplication of a sparse matrix,
 // and finds multiple solution vectors (kernels) where M * x = 0 mod 2.

 std::vector<std::vector<int>> dependencies; // List of dependencies
 // Dummy data
 dependencies.push_back({0});

 std::cout << "[Phase 4] Found " << dependencies.size() << " linear dependencies (perfect squares)." << std::endl;
 return dependencies;
 }
};

// ============================================================================
// Phase 5: Algebraic Square Root
// ============================================================================
class AlgebraicSquareRoot {
public:
 void compute_and_factor(const std::vector<Relation>& relations, const std::vector<int>& dep, const cpp_int& N) {
 std::cout << "[Phase 5] Algebraic Square Root computation..." << std::endl;

 // 1. Compute the rational side square root V (simple integer arithmetic)
 cpp_int V = 1;
 // V = sqrt( prod(a - bm) ) mod N

 // 2. Compute the algebraic side square root gamma (Montgomery's method, etc.)
 // Find an element gamma in the huge algebraic field O_K, and map it to the real world using the homomorphism map phi
 // Y = phi(gamma) mod N
 cpp_int Y = 1;

 // Assuming that sequences of Quadratic Characters were added in Phases 2 and 4
 // to bypass the obstruction of the ideal class group and the unit group.

 std::cout << " -> Homomorphism map phi applied." << std::endl;
 std::cout << "[Phase 5] Calculating GCD(V - Y, N)..." << std::endl;

 cpp_int factor = gcd(V - Y, N); // GCD(X-Y, N)

 if (factor > 1 && factor < N) {
 std::cout << "\n================================================================" << std::endl;
 std::cout << "[SUCCESS] Non-trivial factor found: " << factor << std::endl;
 std::cout << " Other factor: " << N / factor << std::endl;
 std::cout << "================================================================" << std::endl;
 } else {
 std::cout << "[FAILURE] Trivial solution. Trying next dependency..." << std::endl;
 }
 }
};

// ============================================================================
// Main Execution Pipeline
// ============================================================================
int main() {
 std::cout << "================================================================" << std::endl;
 std::cout << " [SOTA GNFS] General Number Field Sieve Engine (Boost C++) " << std::endl;
 std::cout << "================================================================" << std::endl;

 // Huge composite number N to factor, such as RSA-270
 cpp_int N("233108530344407544527637656910680524145619812480305449042948611968495918245135782867888369318577116418213919268572658314913060672626911354027609793166341626693946596196427744273886601876896313468704059066746903123910748277606548649151920812699309766587514735456594993207");

 // Degree of the polynomial (normally select degree 5-6 for numbers over 130 digits)
 int degree = 6;

 // Initialize pipeline
 PolynomialSelector poly_select(degree);
 LatticeSieve sieve(10000000, 20000000); // Actual bounds are tens of millions to hundreds of millions
 Filter filter;
 LinearAlgebraGF2 linalg;
 AlgebraicSquareRoot sqrt_step;

 auto start_time = std::chrono::high_resolution_clock::now();

 // 1. Polynomial selection
 poly_select.select(N);

 // 2. Sieving process
 sieve.generate_factor_bases();
 std::vector<Relation> relations = sieve.sieve(poly_select);

 // 3. Filtering (matrix compression)
 filter.reduce_matrix(relations);

 // 4. Linear algebra (nullspace search over GF(2))
 std::vector<std::vector<int>> dependencies = linalg.solve_nullspace(relations);

 // 5. Algebraic square root computation and GCD
 for (const auto& dep : dependencies) {
 sqrt_step.compute_and_factor(relations, dep, N);
 }

 auto end_time = std::chrono::high_resolution_clock::now();
 std::chrono::duration<double> elapsed = end_time - start_time;
 std::cout << "\n[System] SOTA GNFS Pipeline completed in " << elapsed.count() << " seconds." << std::endl;

 return 0;
}

Now, how does this code destroy the cryptographic wall? I will break down and explain the meticulous algorithm and advanced mathematics phase by phase.

2. The Final Goal of GNFS: $X^2 \equiv Y^2 \pmod N$

The goal that not only GNFS but most modern large integer factorization algorithms aim for is to find a non-trivial pair $(X, Y)$ that satisfies the following congruence:

$$X^2 \equiv Y^2 \pmod N$$

This equation means that “the remainders of $X^2$ and $Y^2$ divided by $N$ are equal”. If we transform this: $X^2 - Y^2 \equiv 0 \pmod N$ In other words, $(X-Y)(X+Y)$ becomes a multiple of $N$.

If $X \not\equiv \pm Y \pmod N$ (a non-trivial solution), then between $(X-Y)$ and $N$, there exists a “common divisor greater than 1 and less than $N$”. Here, if we compute $\gcd(X-Y, N)$ using the Euclidean algorithm, the prime factors of $N$ can be easily found.

However, finding these $X$ and $Y$ is like looking for a needle in a desert. Thus, GNFS takes the genius approach of creating two worlds, the “real integer world” and the “algebraic field of polynomials world”, and distributing the computation.

3. Phase 1: Polynomial Selection

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


class PolynomialSelector {
 // ...
 void select(const cpp_int& N) {
 // Calculation of m = N^(1/d) and base-m expansion
 // ...
 for (int i = 0; i <= degree; ++i) {
 f[i] = temp % m;
 temp /= m;
 }
 g = {-m, 1}; // g(x) = x - m
 }
};

The first step of GNFS is to create a “magic polynomial” to bridge the two worlds. For a huge number $N$, we choose an integer $m$. Usually, it is chosen such that $m \approx N^{1/d}$ (in the code, a polynomial of degree $d=6$ is assumed).

Then, $N$ is expanded in base $m$, and its coefficients are used to construct the polynomial $f(x)$.

$$N = c_d m^d + c_{d-1} m^{d-1} + \dots + c_1 m + c_0$$ $$f(x) = c_d x^d + c_{d-1} x^{d-1} + \dots + c_1 x + c_0$$

This polynomial $f(x)$ has the extremely important property that “substituting $m$ for the variable $x$ evaluates exactly to $N$ ($f(m) = N$)”. In other words, $f(m) \equiv 0 \pmod N$. The rational side polynomial is defined as $g(x) = x - m$.

This strongly connects the “algebraic field world $\mathbb{Z}[\alpha]$” ruled by the root $\alpha$ of $f(x)=0$, and the normal “rational (integer) world $\mathbb{Z}$”, via a “ring homomorphism” of $x \to m$.

In state-of-the-art systems like CADO-NFS, it takes months to search for the “most convenient polynomial $f(x)$” using KleinJung’s algorithm and the LLL lattice basis reduction algorithm, so that the coefficients of the polynomial do not become excessively large, and primes are likely to appear (become smooth) in the subsequent steps.

4. Phase 2: Special-q Lattice Sieving

1
2
3
4
5
6
7
8
9


class LatticeSieve {
 // ...
 std::vector<Relation> sieve(const PolynomialSelector& poly) {
 // ...
 // Maps (a, b) pairs to lattices for each special prime q,
 // and executes a sieve that maximizes cache efficiency.
 // ...
 }
};

Once the two worlds are prepared, the next step is to search for “smooth numbers (numbers composed entirely of small prime factors)” in both worlds. An infinite number of integer pairs $(a, b)$ are generated, and the following two values are calculated:

Rational side value: $a - bm$
Algebraic side norm: $b^d f(a/b)$

The goal of GNFS is to collect tens of millions to hundreds of millions of these “pairs (Relations) where both the rational side and algebraic side values can be completely factored into only small prime factors”.

In the early GNFS, a “Line Sieve” was used, lining up $(a, b)$ on the $xy$ plane and sequentially dividing them by primes from the edges. However, this caused frequent cache misses due to accessing various parts of memory, and its weakness was being extremely slow.

Therefore, the current state-of-the-art code uses the “Special-q Lattice Sieve” method. By fixing a moderately large prime $q$, we restrict the calculation targets to only “pairs of $(a, b)$ where the algebraic side value is always divisible by $q$”. Since $(a, b)$ satisfying this condition form a “lattice” on the plane, the jump width of calculated addresses becomes constant, fitting perfectly into the CPU’s L1/L2 cache. With the introduction of this lattice sieving, the calculation speed of GNFS improved dramatically.

5. Phase 3: Filtering

1
2
3
4
5
6
7


class Filter {
public:
 void reduce_matrix(std::vector<Relation>& relations) {
 // 1. Singleton removal (removing relations with primes that appear only once)
 // 2. Clique merging (merging relations to make a sparse matrix denser)
 }
};

Hundreds of millions of relations collected over months by computers around the world in Phase 2. However, if this is thrown as-is into the next “step of solving simultaneous equations (matrix calculation)”, the supercomputer’s memory will blow up.

Thus, an ultra-compression process of the matrix called Filtering is performed.

Singleton removal Suppose a huge prime $p$ appeared “only once” in hundreds of millions of relations. Since our goal is to “make the exponents of all primes even (multiples of 2)”, a prime that appears only once can never be made even. Therefore, relations containing that prime are immediately removed (purged) as “useless garbage”. As this happens in a chain reaction, the data that had hundreds of millions of rows is rapidly reduced.
Clique merging Furthermore, by multiplying (adding) relations that share specific primes together, it reduces the number of rows while compressing the sparse (empty) matrix into a denser state (using a method similar to clique search in graph theory).

With this optimization, the massive sparse matrix is dramatically compressed to a computable size.

6. Phase 4: Linear Algebra over GF(2) (Block Wiedemann Method)

1
2
3
4
5
6
7


class LinearAlgebraGF2 {
public:
 std::vector<std::vector<int>> solve_nullspace(const std::vector<Relation>& relations) {
 // Iterates matrix-vector multiplication of a sparse matrix,
 // and finds multiple solution vectors (kernels) where M * x = 0 mod 2.
 }
};

Finally, the core of the puzzle. We multiply the collected relations to find the “combination where the exponents of all prime factors become even”.

Mathematically, this means using a huge matrix $M$ whose elements are the “even/odd (i.e., 0 or 1)” of the exponent of each prime, and a vector $x$ representing which relations to use, and finding the solution vector $x$ (nullspace/kernel) such that: $M \cdot x \equiv 0 \pmod 2$

We must solve a system of simultaneous equations for a matrix of an enormous size, millions of rows by millions of columns. With standard Gaussian elimination, the computational complexity would be $O(N^3)$, and the calculation wouldn’t finish until the end of the universe.

Thus, the “Block Wiedemann method” is adopted in state-of-the-art implementations. This is a type of Krylov subspace method that leverages the fact that the matrix $M$ is “extremely sparse (mostly 0s)” to derive a solution by iteratively performing matrix-vector multiplications. Unlike the older Block Lanczos method, the Block Wiedemann method can completely divide the computational process across multiple clusters, making it overwhelmingly powerful for parallel computing in modern distributed cloud computing and supercomputers.

7. Phase 5: Algebraic Square Root and Cryptographic Collapse

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


class AlgebraicSquareRoot {
public:
 void compute_and_factor(...) {
 // 1. Compute the rational side square root V
 cpp_int V = 1;

 // 2. Compute the algebraic side square root gamma
 cpp_int Y = 1;

 // ...
 cpp_int factor = gcd(V - Y, N); // GCD(X-Y, N)
 }
};

Through the matrix calculation in Phase 4, we obtained a “set of relations $S$ whose product yields even powers for all prime factors”. With this, we can construct a “square” in both the rational side and the algebraic side worlds.

For the rational side, it’s just integer multiplication, so computing the square root $V$ is easy.

$$V^2 = \prod_{S} (a - bm)$$

However, the real hell lies on the “algebraic side”. In the algebraic field world $\mathbb{Z}[\alpha]$, since the uniqueness of prime factorization does not hold, calculations have been performed using ideals. What was guaranteed by the matrix calculation is only that it becomes a “square of an ideal”, and it is not guaranteed that it becomes a “square of an element ($\gamma^2$)”.

Here stands a formidable wall in algebraic number theory: the “obstruction of the ideal class group” and the “obstruction of the unit group”. In GNFS, we use the magic of “Quadratic Characters” to break through this wall. Columns of quadratic residues (Legendre symbols) for several tens of special prime ideals are secretly added in advance to the matrix in Phase 4. As a result, the found set $S$ ignores the obstructions with an overwhelming probability and successfully forms the “true square of an element $\gamma^2$”.

The work of finding $\gamma$ (algebraic square root) is computed using highly complex algorithms such as Montgomery’s method.

And finally, we warp the algebraic side square root $\gamma$ into the real world (by substituting $m$ for $x$) via the ring homomorphism $\phi$, yielding $Y$. If we simply set the rational side $V$ as $X$, the absolute equation we have been pursuing is finally complete.

$$X^2 \equiv Y^2 \pmod N$$

All that is left is to compute $\gcd(X-Y, N)$. Running through the 0.001-second process, the moment a non-trivial factor is printed on the screen, the proudly impregnable RSA cryptography completely collapses.

Conclusion

GNFS is not just a programming technique. It is a crystal of human intellect that has wrestled down the “depths of pure mathematics” like abstract algebra, ring theory, and ideal class groups using “extreme engineering” like supercomputer distributed architectures and cache optimizations.

The chat messages and credit card information we casually transmit are protected upon such astronomical mathematical defense and offense.

Through this C++ framework, I hope you have felt the “romance of mathematics and computers” behind state-of-the-art cryptanalysis algorithms.

A 400-Year Mystery! What is the Kepler Conjecture? Explaining the Romance of Mathematics Learned from Packing Watermelons

Mon, 21 Jul 2025 22:53:03 +0900

A Simple Explanation of Kepler’s Conjecture! The Best Way to Pack Watermelons Tightly

Hello, this is kenji!

Today, I’d like to explain a mathematical concept with a somewhat difficult-sounding name, “Kepler’s Conjecture,” as simply as possible.

At first glance, it seems like a very niche topic, but it’s actually a quite familiar problem: “How can we pack watermelons as tightly as possible in a box?” It relates to convenience store refrigerators and how cargo is loaded.

And in the world of mathematics, it’s a super romantic story because it could not be proven for over 400 years.

Let’s get started!

Who is Kepler anyway?

First of all, who is the “Kepler” in “Kepler’s Conjecture”?

This is the name of a German astronomer and mathematician, Johannes Kepler.

He was a truly amazing person. For example, he’s the one who discovered that planetary orbits are elliptical. It’s common knowledge now, but back then (around 1600), even the heliocentric theory wasn’t widely believed.

Galileo = The genius of observation
Kepler = The genius who explained the universe with mathematical formulas
Newton = The super genius who compiled those theories into physical laws

In that kind of positioning, Kepler was a pioneer of “explaining the universe with mathematical formulas.”

And then, one day, he brought this up.

What is Kepler’s Conjecture?

Put simply, “Kepler’s Conjecture” is about this:

When packing spheres of the same size (like watermelons or oranges) into a box, which arrangement packs them the tightest?

That’s the problem.

And what Kepler conjectured in the 1600s was:

Isn’t it most efficient to stack them in a triangle-like shape, just like how oranges are displayed at a greengrocer?

That was it.

To say this more mathematically, it’s called the “Sphere Packing Problem,” which is the problem of finding the densest way to pack spheres into space.

Kepler conjectured that an arrangement called “Face-Centered Cubic (FCC)” structure (like stacking watermelons in a triangle) is the most efficient.

And, while everyone thought, “That certainly seems right,” it was incredibly difficult to mathematically prove that “This is absolutely the best!”

400 Years Until It Was Proven!?!

So, when was it actually proven?

Surprisingly, around 1998 to 2005. In other words, no one could prove it for nearly 400 years. That’s crazy.

Moreover, it was proven by an American mathematician named Thomas Hales.

He decided it was impossible to do by hand calculations alone, and proved it using computers. However, the proof was so complex that a problem arose: humans couldn’t properly check it!

This sparked a huge debate in the mathematical world: “So, can we really trust a proof done by a computer?”

In the end, the proof that said, “We have rigorously verified it, including the computer part!” (formally known as a “formal proof”) was completed in 2014.

In other words, it took over 400 years to prove that Kepler’s intuition was right. What a romantic story.

It’s Used a Lot in Real Life Too

You might think that “how to arrange spheres” is just a topic for math nerds, but it’s actually super practical.

Efficiently packing drinks in a convenience store refrigerator
Loading canned goods onto transport pallets
Applied to communications (like digital signal compression)
Used in 3D printers and the design of crystal structures

And so on, it’s a theory that’s actually heavily used behind the scenes in our daily lives.

Summary: Human Intuition is Amazing

So, let’s summarize.

Kepler’s Conjecture = The problem of how to pack spheres as tightly as possible
Mr. Kepler = The mathematician and astronomer who discovered planetary orbits
Conjectured in the 1600s! But proven in the 2000s!
It’s highly applied in real life!

And the most interesting part is that “the way greengrocers intuitively stacked things was mathematically the strongest.”

In short, human intuition is amazing. And the fact that it can take 400 years to prove it is such a romantic story.

If you’re interested, please try searching for things like “Thomas Hales”, “Formal Proof”, and “Sphere Packing”. It’s incredibly fascinating when you dig deeper into it.

See you!

[PR]

Buy Now

What is the Collatz Conjecture? Verifying a Mathematical Unsolved Problem Where Any Number Eventually Reaches 1 in Python

Tue, 15 Jul 2025 18:03:03 +0900

“Any number will eventually become 1”? ── Playing with the Collatz Conjecture

Hello! It’s kenji.

All of a sudden, doesn’t it sound a bit mysterious when you hear about “a rule where any number eventually becomes 1”?

For example, 19, or 87, or even 1000000. If you manipulate the number according to a certain rule, it somehow converges to “1” in the end.

Such a dream-like story is the Collatz Conjecture.

What exactly is the Collatz Conjecture?

First, let’s introduce the rules.

Start: Pick any positive integer
Operation:
- If it’s even → Halve it (n → n / 2)
- If it’s odd → Multiply by 3 and add 1 (n → 3n + 1)

The conjecture is that if you keep repeating this, any number will eventually reach 1.

For example, starting with 6:

1

6 → 3 → 10 → 5 → 16 → 8 → 4 → 2 → 1

It properly became “1”. Welcome back!

Let’s try it with code: Collatz in Python

Now, in times like this, it’s faster to test it with code! Let’s output the “Collatz sequence” in Python.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


def collatz(n):
 steps = [n]
 while n != 1:
 if n % 2 == 0:
 n = n // 2
 else:
 n = 3 * n + 1
 steps.append(n)
 return steps

# Example: Let's start with 19
print(collatz(19))

When you run it:

1

[19, 58, 29, 88, 44, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1]

It beautifully reaches 1. It takes quite a detour, but reaches the goal securely at the end!

By the way, even if you start with 27, it reaches 1 in the same way.

1

print(collatz(27))

When you run it:

1
2
3
4
5
6
7
8


[27, 82, 41, 124, 62, 31, 94, 47, 142, 71, 214, 107, 322, 161, 484, 242,
121, 364, 182, 91, 274, 137, 412, 206, 103, 310, 155, 466, 233, 700, 350,
175, 526, 263, 790, 395, 1186, 593, 1780, 890, 445, 1336, 668, 334, 167,
502, 251, 754, 377, 1132, 566, 283, 850, 425, 1276, 638, 319, 958, 479,
1438, 719, 2158, 1079, 3238, 1619, 4858, 2429, 7288, 3644, 1822, 911,
2734, 1367, 4102, 2051, 6154, 3077, 9232, 4616, 2308, 1154, 577, 1732,
866, 433, 1300, 650, 325, 976, 488, 244, 122, 61, 184, 92, 46, 23, 70, 35,
106, 53, 160, 80, 40, 20, 10, 5, 16, 8, 4, 2, 1]

Surprisingly, it takes 111 steps!

Moreover, there are moments where it swells up to over 9000 along the way. It’s a pattern that takes a massive detour before hitting the goal.

So, what’s so amazing about it?

What’s amazing about this conjecture is,

Even though it hasn’t been proven, it seems to become 1 no matter what number you try

That’s the point.

Eh? What about 1 trillion, or 10 quadrillion…?

For those who thought that, you’re sharp. Actually, it has been verified using computers up to about “2 to the power of 68”, and all of them have reached 1. Unbelievable…

However, it hasn’t been theoretically proven that “it works for all of them”. This is what’s called an “unsolved problem” in the world of mathematics.

Why does it become “1”? An approach from probability theory (Mathematical background)

It seems like magic that any number eventually becomes 1, but from a probabilistic perspective, there is a reasonable logic of “well, that seems likely to happen.”

If you apply 3n + 1 to an odd number $n$, the answer is always an even number. Therefore, in the next step, it will definitely be divided by 2, practically becoming $\frac{3n + 1}{2} \approx 1.5n$.

Then, the probability that the resulting number is even again is $\frac{1}{2}$. If it’s even, it’s further divided by 2 to become $0.75n$, which is smaller than the original number.

Although not mathematically rigorous, it is known that taking the geometric mean of the “multiplier” when jumping from one odd number to the next odd number gives approximately $\frac{3}{4}$ times (a heuristic probability model). In other words, since the value tends to shrink on average, it eventually falls as if being sucked into 1.

What if we slightly change the rules? (Comparison with other conjectures)

“Well then, what if we multiply by 5 instead of 3?” you might want to consider. Actually, this is known as the $5n + 1$ problem, and in this case, not all numbers converge to 1.

In the case of $5n + 1$, it has been confirmed that multiple different loops (cycles) exist, and it’s also been pointed out that there might be numbers that continue to grow infinitely (divergence). Also, in the case of the $3n - 1$ problem, aside from the “$1 \to 2 \to 1$” loop, there is another loop like “$5 \to 14 \to 7 \to 20 \to 10 \to 5$”.

You can see how the property of the Collatz Conjecture, “everything converges to 1 (the $4 \to 2 \to 1$ loop)”, rests upon an incredibly exquisite balance.

Humanity’s milestone ①: The limits of brute force by computers

Currently, mathematicians and computer science enthusiasts around the world are continuously calculating the Collatz Conjecture, making full use of distributed computing (projects that combine the computational power of PCs worldwide) and GPUs.

As of 2020, computers have confirmed that the Collatz Conjecture holds true (eventually becomes 1) for all initial values up to a staggering $2^{68}$ (about 295,000 trillion).

However, in the world of mathematics, you can’t say, “Since we’ve checked up to 295 quadrillion, it must be true for everything.” From the perspective of the infinite sea of numbers, even $2^{68}$ is nothing more than “the first drop.”

Humanity’s milestone ②: Undecidability and Terence Tao’s breakthrough

In response to the question “Why can’t anyone prove it?”, brilliant British mathematician John Conway proved in 1972 that a slightly extended problem of the Collatz Conjecture is “Turing complete” (undecidable). This is a terrifying fact deeply related to the foundations of computer science, meaning that depending on the rules, “an algorithm to determine whether it reaches 1 does not exist in principle.” The Collatz Conjecture itself might even be an unprovable proposition within the framework of modern mathematics.

However, in 2019, a major breakthrough finally occurred. Terence Tao, one of the greatest genius mathematicians of our modern era, proved that “(while we can’t strictly say for all) for almost all initial values, the Collatz sequence eventually reaches a value much smaller than the original number” by utilizing partial differential equations and probability theory.

Although this is not a complete proof that “everything becomes 1,” it caused a stir in the global mathematical community as the historical milestone where humanity came closest to the truth of the Collatz Conjecture.

Who is Mr. Collatz?

Now, reading up to this point, you might think, “Who is Collatz anyway?” Let me properly introduce him!

Name: Lothar Collatz
Nationality: Germany
Lifespan: 1910 - 1990
Title: Mathematician (Active in the fields of functional analysis and number theory)

He proposed this conjecture in 1937, and for over 80 years since then, no one has been able to prove or disprove it.

By the way, this problem is so simple yet so profoundly deep, that even Paul Erdős (a super famous mathematician) allegedly said this:

“Mathematics may not be ready for such problems.”

In other words, the theory is that humanity’s mathematics hasn’t caught up to this mystery yet…

No “difficult formulas” are necessary

The great thing about the Collatz Conjecture is that anyone can play with it.

You can do it if you have pen and paper. If you write code in Python, you can test it automatically. And yet, cutting-edge mathematicians are seriously challenging it.

Doesn’t that somehow make you excited?

Bonus: Code to test it all at once

I’ll also leave a piece of code here to try out various numbers all together.

1
2
3


for n in range(1, 21):
 steps = collatz(n)
 print(f"{n}: {steps} (Number of steps: {len(steps)-1})")

This will spit out the Collatz sequences for “1 to 20” all at once.

Conclusion: This world is mysterious after all

So, there you have it, the Collatz Conjecture.

Even though it’s incredibly simple
No one can prove it
It’s a huge problem in the mathematical world

It’s a phenomenon that’s essentially a block of mystery.

Even beginners in programming can try it, so please do play around with it!

What is the P≠NP Conjecture? An Easy-to-Understand Explanation of the Unsolved Problem in Computational Complexity Theory and the Difference Between Class P and NP

Wed, 11 Sep 2024 02:22:39 +0900

Overview

1

Class P is the class of problems that are decidable in polynomial time by a deterministic Turing machine. Class NP is the class of problems for which the correctness of a witness (evidence that the answer is Yes) can be verified in polynomial time when given the witness. Since problems decidable in polynomial time are also verifiable in polynomial time, it is obvious that P ⊆ NP. However, it is not clear whether P is a proper subset of NP. Although there is no proof yet, many researchers believe that P ≠ NP. This conjecture that class P and class NP are not equal is known as the "P ≠ NP conjecture".

Reference site : https://daigakudenki.com/np-hard/

Don't Give in to Criticism! Tips for Living Strong Learned from Einstein's Quotes

Sat, 24 Aug 2024 18:38:47 +0900

Hello!

Today, I would like to talk about a quote by Albert Einstein.

The World Doesn’t Appreciate Your Good Deeds?

` The world will never appreciate the million good things you do. But it will criticize the one wrong thing you do… But do not be discouraged. “Always rise above all laughter and criticism. Stay strong.”

Albert Einstein `

This is a quote attributed to Einstein. It sharply points out the harshness of society and the bias in people’s evaluations that we feel every day.

Still, Keep Rising

“Always rise above all laughter and criticism. Stay strong.”

This following sentence can be said to be Einstein’s answer to the harsh reality mentioned above. It teaches us the importance of not yielding to criticism or ridicule, and continuing to move forward believing in ourselves.

Applying Einstein’s Teachings in Daily Life

This quote is deeply related to our daily lives. For example, there are times when your efforts at work go unrecognized, and only trivial mistakes stand out. Also, your statements on social media might be taken out of context and criticized.

At such times, try to remember this quote by Einstein. It is important to hold on to your own beliefs rather than being swayed by the evaluations of others.

Summary

This quote by Einstein teaches us the importance of “living believing in yourself without being swayed by the evaluations of others”. Let’s have the courage to pursue the path we believe in, without yielding to criticism or ridicule.

Finally, Einstein’s words once again.

“Always rise above all laughter and criticism. Stay strong.”

I hope this quote resonates with your heart and becomes a catalyst for you to take a positive step forward.

Mathematica Introduction: How to Use Basic Commands for Equations, Calculus, etc.

Thu, 25 Jul 2024 01:36:19 +0900

Introduction to Mathematica

Solve an equation

1

Solve[x^2 - 3 x + 2 == 0, x]

Output

1

{{x -> 1}, {x -> 2}}

Solve an equation within integers

1

Solve[x^2 - 3 x + 2 == 0 && 0 <= x <= 2, x, Integers]

Output

1

{{x -> 1}, {x -> 2}}

Solve simultaneous equations

1

Solve[{x + y == 3, x - y == 1}, {x, y}]

Output

1

{{x -> 2, y -> 1}}

Solve an inequality

1

Reduce[x^2 - 3 x + 2 > 0, x]

Output

1

x < 1 || x > 2

Differentiate

1

D[x^2, x]

Output

1

2 x

Integrate

1

Integrate[x^2, x]

Output

1

x^3/3

Find a limit

1

Limit[1/x, x -> 0]

Output

1

Infinity

Find a series

1

Sum[1/n^2, {n, 1, Infinity}]

Output

1

π^2/6

Create a matrix

1

m = {{1, 2}, {3, 4}}

Find the product of matrices

1

m . m

Output

1

{{7, 10}, {15, 22}}

Find the inverse matrix

1

Inverse[m]

Output

1

{{-2, 1}, {1.5, -0.5}}

Find eigenvalues and eigenvectors

1

Eigensystem[m]

Output

1

{{5, 0}, {{1, 1}, {1, -1}}}

Find the inner product of vectors

1

{1, 2} . {3, 4}

Output

1

11

Find the cross product of vectors

1

Cross[{1, 2, 3}, {4, 5, 6}]

Output

1

{-3, 6, -3}

Find the magnitude of a vector

1

Norm[{1, 2, 3}]

Output

1

√14

Find the angle between vectors

1

ArcCos[{1, 2} . {3, 4}/(Norm[{1, 2}] Norm[{3, 4}])]

Output

1

ArcCos[11/(√5 √25)]

Find the projection of a vector

1

{1, 2} . {3, 4}/Norm[{3, 4}] {3, 4}/Norm[{3, 4}]

Output

1

{11/5, 22/5}

Find the rotation of a vector

1

RotationMatrix[π/2].{1, 0}

Output

1

{0, 1}

Find the translation of a vector

1

TranslationTransform[{1, 2}][{3, 4}]

Output

1

{4, 6}

Find the scaling of a vector

1

ScalingTransform[{2, 3}][{1, 1}]

Output

1

{2, 3}

Find the reflection of a vector

1

ReflectionTransform[{1, 1}][{1, 1}]

Output

1

{0, 0}

Generate random numbers

1

RandomReal[]

Output

1

0.123456

Generate random numbers (integers)

1

RandomInteger[]

Output

1

123456

Generate random numbers (range specified)

1

RandomReal[{1, 10}]

Output

1

5.6789

Generate random numbers (integers, range specified)

1

RandomInteger[{1, 10}]

Output

1

5

Generate random numbers (distribution specified)

1

RandomVariate[NormalDistribution[0, 1]]

Output

1

0.123456

Generate random numbers (distribution specified, number specified)

1

RandomVariate[NormalDistribution[0, 1], 10]

Output

1

{0.123456, 0.234567, ..., 0.987654}

Generate random numbers (distribution specified, number specified, seed specified)

1
2


SeedRandom[12345]
RandomVariate[NormalDistribution[0, 1], 10]

Output

1

{0.123456, 0.234567, ..., 0.987654}

Apply a function to array elements

1
2
3


Map[Sqrt, {1, 4, 9}]
Sqrt /@ {1, 4, 9}
Map[#^(1/2)&, {1, 4, 9}]

Output

1

{1, 2, 3}

Define a function using lambda expressions

1
2


f = Function[x, x^2]
f[3]

Output

1

9

Compose functions

1
2
3
4


f = Function[x, x^2]
g = Function[x, x + 1]
h = Function[x, f[g[x]]]
h[3]

Output

1

16

Reference the previous calculation result

1

% + 1

Output

1

17

Pure functions

1

(#+3)&[5]

Output

1

8

Extract from an array

1
2


Select[{1, 2, 3, 4, 5}, EvenQ]
Select[{1, 2, 3, 4, 5}, Mod[#,2]==0&]

Output

1

{2, 4}

What is the Sieve of Eratosthenes? Algorithm and Implementation for Enumerating Primes Under 1000

Sun, 09 Apr 2023 12:54:24 +0900

What is the Sieve of Eratosthenes?

The Sieve of Eratosthenes is an algorithm for finding all prime numbers up to any given limit. The algorithm is simple and can be implemented with the following steps:

Create an array of boolean values with N elements, and initialize all elements to true.
Set the 0th and 1st elements of the array to false (because 0 and 1 are not prime numbers).
If the 2nd element of the array is true, output 2 as a prime number.
Set all multiples of 2 from $2^2$ onwards in the array to false.*
If the 3rd element of the array is true, output 3 as a prime number.
Set all multiples of 3 from $3^2$ onwards in the array to false.
Repeat the same process for the 4th, 5th, …, Nth elements.

*The reason for targeting elements from the square of the number onwards to become false is because the numbers smaller than the square have already been processed (enumeration is complete).

Implementation in Rust

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


fn main() {
 let n = 1000;
 let mut is_prime = vec![true; n+1];
 is_prime[0] = false;
 is_prime[1] = false;
 for i in 2..=n {
 if is_prime[i] {
 println!("{}", i);
 let mut j = i * i;
 while j <= n {
 is_prime[j] = false;
 j += i;
 }
 }
 }
}

Slightly Faster Version

Considering the following points, we implement a slightly faster version:

Instead of initializing the array with true, initialize it with false (this is faster).
Since multiples of 2 are not prime numbers, omit the process of setting the elements of multiples of 2 to false.
There is no need to loop up to n; by enumerating primes up to the square root of n, you can enumerate primes up to n.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


fn main() {
 let n = 1000;
 let mut is_prime = vec![false; n+1];
 is_prime[2] = true;
 for i in (3..=n).step_by(2) {
 is_prime[i] = true;
 }
 for i in 3..=((n as f64).sqrt() as usize) {
 if is_prime[i] {
 let mut j = i * i;
 while j <= n {
 is_prime[j] = false;
 j += i * 2;
 }
 }
 }
 for i in (2..=n).filter(|&x| is_prime[x]) {
 println!("{}", i);
 }
}

References

Sieve of Eratosthenes

How to Draw Graphs with Python and matplotlib [Google Colab Compatible]

Sun, 09 Apr 2023 01:02:19 +0900

Requirements

Google Account

Steps

Access https://colab.research.google.com/
Select “File” -> “New notebook”
Paste and run the following code

1
2
3
4
5
6
7


import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0, 2*np.pi, 500)
plt.plot(x, np.sin(x), label="sin curve")
plt.plot(x, np.cos(x), label="cos curve")
plt.legend() # Show legend
plt.show()

Execution Result

References

matplotlib.pyplot — Matplotlib 3.5.3 documentation

Genius Mathematician Protagonists! Recommended Masterpiece Movies and Stories of Codebreaking

Mon, 05 Sep 2022 10:41:00 +0900

Here are my 3 recommended movies featuring mathematician protagonists.

A Beautiful Mind

A film depicting the life of a real-life American genius mathematician. A surprising twist at the end. Russell Crowe’s performance in the lead role is excellent.

Enigma

A 2001 film adaptation of Robert Harris’s novel “Enigma”. — Wikipedia

A story about decrypting the codes generated by the German Enigma cipher machine.

The Imitation Game

Another story about cryptanalysis dealing with the Enigma. I love the atmosphere of that era.

The Man Who Knew Infinity

A film depicting the interaction between Cambridge University mathematician G.H. Hardy and Indian mathematician Srinivasa Ramanujan. You can feel the beauty of mathematics.

The Theory of Everything

A film depicting the life of Dr. Stephen Hawking.

Math on kenji.blog

【For Beginners】 Deciphering the Mathematical Structure of the Transformer Model

Introduction: Why Learn the Mathematics of Transformers?

1. Limitations of Conventional Methods and the Innovativeness of the Transformer

2. Overall Architecture of the Transformer

3. Word Vectorization and Positional Encoding

3.1 Mathematics of Input Embedding

3.2 The Need for Positional Encoding and its Formulas

Why Use Sine and Cosine?

4. The Profound Mathematics of Self-Attention

4.1 Generation of Matrices $Q, K, V$

4.2 Calculation of Attention Scores (Dot Product)

4.3 Scaling (Scale)

4.4 Probabilization by Softmax Function

4.5 Weighted Sum of Value

5. Multi-Head Attention

6. Feed-Forward Neural Network (FFN)

7. Residual Connection and Layer Normalization

7.1 Residual Connection ($x + \text{Sublayer}(x)$)

7.2 Mathematics of Layer Normalization

8. Decoder-Specific Structures: Masked Attention and Cross-Attention

8.1 Masked Multi-Head Attention

8.2 Encoder-Decoder Cross-Attention

9. Computational Complexity and Mathematics of Modern Optimization

10. Implementation Concept (PyTorch-style Pseudocode)

Conclusion: The Shape of “Intelligence” Seen Through Mathematical Formulas

References

Implementing General Number Field Sieve (GNFS) in C++ | Mechanism of RSA Cryptanalysis Algorithm

[Complete Anatomy] Understanding the Strongest Cryptanalysis Algorithm “GNFS” by Implementing it in C++

1. GNFS State-of-the-Art Logic Framework (Full Source Code)

2. The Final Goal of GNFS: $X^2 \equiv Y^2 \pmod N$

3. Phase 1: Polynomial Selection

4. Phase 2: Special-q Lattice Sieving

5. Phase 3: Filtering

6. Phase 4: Linear Algebra over GF(2) (Block Wiedemann Method)

7. Phase 5: Algebraic Square Root and Cryptographic Collapse

Conclusion

A 400-Year Mystery! What is the Kepler Conjecture? Explaining the Romance of Mathematics Learned from Packing Watermelons

A Simple Explanation of Kepler’s Conjecture! The Best Way to Pack Watermelons Tightly

Who is Kepler anyway?

What is Kepler’s Conjecture?

400 Years Until It Was Proven!?!

It’s Used a Lot in Real Life Too

Summary: Human Intuition is Amazing

What is the Collatz Conjecture? Verifying a Mathematical Unsolved Problem Where Any Number Eventually Reaches 1 in Python

“Any number will eventually become 1”? ── Playing with the Collatz Conjecture

What exactly is the Collatz Conjecture?

Let’s try it with code: Collatz in Python

So, what’s so amazing about it?

Why does it become “1”? An approach from probability theory (Mathematical background)

What if we slightly change the rules? (Comparison with other conjectures)

Humanity’s milestone ①: The limits of brute force by computers

Humanity’s milestone ②: Undecidability and Terence Tao’s breakthrough

Who is Mr. Collatz?

No “difficult formulas” are necessary

Bonus: Code to test it all at once

Conclusion: This world is mysterious after all

Recommended Links (For those interested)

What is the P≠NP Conjecture? An Easy-to-Understand Explanation of the Unsolved Problem in Computational Complexity Theory and the Difference Between Class P and NP

Overview

Don't Give in to Criticism! Tips for Living Strong Learned from Einstein's Quotes

The World Doesn’t Appreciate Your Good Deeds?

Still, Keep Rising

Applying Einstein’s Teachings in Daily Life

Summary

Mathematica Introduction: How to Use Basic Commands for Equations, Calculus, etc.

Introduction to Mathematica

Solve an equation

Solve an equation within integers

Solve simultaneous equations

Solve an inequality

Differentiate

Integrate

Find a limit

Find a series

Create a matrix

Find the product of matrices

Find the inverse matrix

Find eigenvalues and eigenvectors

Find the inner product of vectors