Thoughts, et cetera.

Your CPU Has More Registers Than You'd Think

2026-04-24T00:00:00+05:30

Let’s start with a question: How many registers does your CPU have?

If you’re on a typical AArch64 machine, you’d start by listing the general purpose registers (x0-x31), the SIMD registers (Q0-Q31), zero register (xzr), the stack pointer (SP), program counter (PC) just to name a few. That adds to a total of 66 registers. However, if we were to zoom-in on a die shot for a CPU, we won’t be finding x0, x1 or any of the other registers. Instead, we’ll discover a large register file with hundreds of registers. These are often called “physical registers” and are differentiated from “architectural registers” (x0, x1 …).

This blog post is an inspection of this circuitry from an algorithmic point of view and the compiler optimizations it enables.

Out-Of-Order Execution

Modern, high-performance CPUs execute instructions in out-of-order fashion to exploit instruction-level parallelism. As a result, execution pipelines tend to be multi-ported to support parallel execution, deep and complex.

For example, here’s the execution pipeline of the ARM Neoverse V2 Microarchitecture (PDF):

ARM Neoverse V2 Pipeline

First two stages of the pipeline are in-order, meaning the fetch unit fetches instructions from DRAM in program order and the decode unit goes through these instructions also in order. The decode unit is where all the interesting stuff happens and it’s the subject of this post. Post the decode unit, execution happens out of order by the parallel execution units.

Neoverse, pictured above, has 17 different execution units that all operate in parallel. As should be obvious from the names, branch units handle branch instructions, integer units; divided for single/multi cycle operations handle instructions like add, div and mul.

Decode

We start with the decode unit. The decoder figures out how many different resources (for example, a slot in the Re-Order Buffer) an instruction may need, which execution unit it belongs to and splits an instruction into many micro-ops. For example, the STP instruction of AArch64 is split into two micro-ops: store-address and store-data. Micro-architectures are generally described with an “x-wide” classification. For example, the Neoverse V2 is 4-wide and Apple M1 is 8-wide. The decoder unit is where wideness comes from. 4-wide implies the decoder is capable of dispatching 4 micro-ops per cycle.

Rename

Following the decoder is the rename/map unit. The rename unit maps/allocates a physical register for every architectural register. It is responsible for removing false dependencies from a set of instructions so that they can be executed out-of-order. Consider this snippet of AArch64 assembly:

1 add x3, x2, x1
2 sub x4, x3, x1
3 add x3, x5, x1
4 mul x6, x8, x7

Instruction 2 is clearly dependent on instruction 1’s results (written in register x3) and instruction 4 depends on instruction 3. At a first glance, it may appear that I3 depends on I1, but this is a case of Write-After-Write (WAW) and is commonly called a “false dependency” as I3 makes no use of the value written by I1. This would not be a problem if I3 used a register other than x3.

We can change our code or let the CPU do this automatically. The registers used in the snippet are “Architectural registers”. In a sense, architectural registers are hypothetical. If we were to zoom in on the die-shot of a CPU, we will not find any registers explicitly named x3. Instead, we find a large register file, with hundreds of registers. These are the “Physical” or real registers. Let’s call them P1, P2 and so on.

The renamer maps registers x1, x2 … to physical registers P1, P2… It also keeps track of when an instruction retires so that the physical register assigned to it can be reclaimed.

The hardware that stores the mappings is called the Register Alias Table (RAT). Roughly speaking, it’s a simple key-value map where the key is an architectural register and the value is a pointer to the physical register in the Physical Register File (PRF).

For the instructions above, this is how the instructions would be renamed as they come out of the decoder.

Step	Arch Instruction	Source Lookups	Dest Map	Physical Instr	Relevant RAT
0	(Initial State)	-	-	-	x1:P1, x2:P2
1	add x3 x2 x1	x2:P2, x1:P1	x3=P10	add P10, P2, P1	x3:P10
2	sub x4 x3 x1	x3:P10, x1:P1	x4=P11	sub P11, P10, P1	x4:P11
3	add x3 x5 x1	x5:P5, x1:P1	x3=P12	add P12, P5, P1	x3:P12
4	mul x6 x8 x7	x8:P8, x7:P7	x6=P13	mul P13, P8, P7	x6:P13

Initially, I1 will cause x2 and x1 to be mapped to P1, P2 and x3 (which is a destination register) will be mapped to P10. I2, which is clearly dependent on I1 (via x3) will correctly recognize the dependency through P10. I3, which had a false dependency (Write-After-Write) on x3 will be rectified as its destination register will be renamed to P12. Finally, I4, which was independent of the other three instructions will have a new physical register assigned to it.

The converted instructions can be seen in the ‘Physical Inst’ column of the table above. Looking at the new instructions, it’s evident that there are no false dependencies present in the instructions now.

Optimizations Enabled By The Renamer

In the previous section, we saw how renaming enabled the CPU to schedule instructions out-of-order. As OoO execution enables ILP which directly affects the throughput, this is the single biggest optimization made possible by the Renamer.

The renamer provides a very important optimization in the form of 0-cycle or issue-less instructions. Consider the following snippet:

orr x1, xzr, x2

this is x1 = 0 | x2 which is essentially an assignment of x2 to x1. Since x1 and x2 are mapped to some physical registers, this instruction can be taken care of during the rename stage by assigning x2’s physical register to x1. This makes it an issue-less instruction or a zero cycle instruction. We can verify this with llvm-mca:

$ echo "orr x1, xzr, x2" | llvm-mca -mcpu=neoverse-v2
Resource pressure by instruction:
[0.0]  [0.1]  [1.0]  [1.1]  [2.0]  [2.1]  [2.2]  [3]    [4.0]  [4.1]  [5]    [6]    [7]    [8]    [9]    [10]   [11]   [12]   [13]   [14]   Instructions:
 -      -      -      -      -      -      -      -      -      -      -      -      -      -      -      -      -      -      -      -     mov  x1, x2

The table above shows that the instruction consumes zero resources from the execution units. It is handled entirely in the rename stage.

Here’s another snippet:

mov x1, #4

This is constant assignment. While some architectures can handle this at the rename stage, it is not always the case. On Neoverse V2, a constant assignment still requires an execution unit:

$ echo "mov x1, #4" | llvm-mca -mcpu=neoverse-v2
Resource pressure by instruction:
... [4.0]  [4.1]  [5]    [6]    [7]    [8]    [9]    [10]   [14]   Instructions:
...  -      -     0.16   0.16   0.17   0.17   0.17   0.17    -     mov  x1, #4

Wait, why does it show 0.16 and 0.17? This is because the instruction can be handled by any of the 6 integer units ([5] through [10]), and llvm-mca shows the average pressure across them.

While a zero cycle instruction sounds exciting, it’s important to note that these do occupy space in the fetch and decode part of the pipeline. They are not really free. Where they do help is freeing up the execution pipeline so that compute isn’t hampered by register-clear/register-move type trivial instructions.

References

These are some of the documents that I used to understand register renaming and other related concepts:

Lockdown Was the Best Thing to Happen to Me

2026-01-30T00:00:00+05:30

I joined university for an engineering degree in the August of 2019. Few months down when the first semester ended and preparation for second began, the news about Coronavirus breakout came out (precisely on 31st of December, 2019). I remember reading about it but not paying much attention initially.

Fast-forward three months and it’s March. The virus is real and it’s spreading fast. My university announced an indefinite vacation. I was watching Linux content here and there (primarily the youtube channel of one Luke Smith. Luke, in his videos, demonstrated all the cool things that a Linux based operating system allows one to do. Things such as setting custom key-bindings, sending mail with just a few key-strokes, ricing your desktop and posting it on r/unixporn, astounding normies with your Vim skillz, you get the point.

Upon return, I got my USB drive, flashed Linux Mint on it, and wiped Windows 10 from the old, dilapidated laptop I had at the time to hopefully breathe a new life into it. And it worked. I was happy and excited to use it. I had things to do with it anyways.

I got started by learning how to use the command line - not just for a purpose - but how to live in it, as that is the primary appeal of a Linux desktop. I started writing simple scripts. It was so exciting to automate things that I started looking to write scripts for things that didn’t need them in the first place. I cannot claim to be a part of the Linux-clique without distro hopping. A couple months in, I switched from Mint to Manjaro to get a flavor of the pacman package manager. A few weeks in to using Manjaro, the pull of Arch Linux became stronger and I gave in. I spent three days installing arch, first making my nvme appear in fdisk, then fixing internet and audio. I had the “I use Arch, btw” privilege. I was happy.

All of this is right before LLM chatbots became mainstream. As there were no LLMs to write my bash one-liners for me, I had to resort to stack overflow, man pages and the wild internet to find what I was looking for. This was web-surfing, as it was done in ye olde times.

Web surfing is perhaps the most engaging, creativity-arousing activity one can do on the Internet. I was doing it. I discovered systems programming, open source, open source history, philosophy, small web like the gemini/gopher boards and tilde.town, obscure/esoteric programming languages, internet forums dedicated to hobbies, new music genres, personal blogs, discussion boards like hacker news and less wrong, academic websites of professors and phd students containing dense information about a subject, and a vast compendium of human knowledge available to be browsed and read.

I went from a coasting-through-life as it comes to aware and learned of what it’s about. Through philosophical forums and books, I learned what thinkers think about. Through programming/tech related forums and blogs, I managed to escape the tutorial hell that newb programmers find themselves into.

I had learnt enough to start a serious project. This was edd, a re-write of the infamous “STANDARD TEXT EDITOR”. Entering into a Comp-Sci degree, web development was the only area I was aware of. This period of stimulating exploration introduced me to the world of systems programming.

As if having linux as the operating system wasn’t enough, I started looking into replacing the bootloader too. Projects like coreboot provide an open alternative to the proprietary bootloader that comes with a laptop. Unfortunately, if your laptop isn’t supported already by coreboot, you’ll have to add support for it. This involves reverse engineering the bootloader, finding out what registers are available etc. I got an old haswell laptop to use as the subject of reverse engineering (as it is easier to rev-eng older intel chips).

Around the same time, due to my work with bootloader stuff and a general exposure to low-level systems, I landed an Internship at a startup working on FPGA based ML Accelerators. I got to work on things and at a level that I would’ve never known existed at the start of this period. This year marks the 7th year of this incredible journey. I will soon be joining a major chip company working on LLVM and compilers. While the Covid lockdown was terrible for the world as a whole and many people suffered, I owe my career to it and the Internet.

Faster Division With Newton-Raphson Approximation

2025-10-13T00:00:00+05:30

Many devices, especially embedded (micro-controllers and the like) do not come with an FPU and the circuitry required for carrying out integer division. In such a case, one looks towards methods of approximating the results of division and storing them in Fixed Point format.

C has standardized support for such an instance via its stdfix library. The ISO Document describes the data types and functions available in stdfix.h.

This post describes the theory, provides a dependency-free C++ implementation of the core algorithm and discusses optimizations to speed it up even further. In that order.

Theory

The problem at hand is that of division:

C = \frac{N}{D}

The first step to solving this is to split the problem in two: Reciprocal calculation, followed by multiplication. This is what it looks like:

C = N \cdot (\frac{1}{D})

It is assumed that the device has a fast multiplication hardware (which is mostly the case). Thus, multiplication can be carried out by the good ol’ mul instruction. Now for the most tricky part - calculating the reciprocal - which is a division operation!

As it turns out, this is a known problem and solutions are approximately as old as the Unix epoch.

Newton-Raphson Method

The Wikipedia article sufficiently describes what and how of the NR Method. I’ll summarize and add some missing context.

“Newton’s Method is a root-finding algorithm which produces successively better approximations to the roots (or zeroes) of a real-valued function.” This is the generic iterative equation according to Newton’s method:

x_{i + 1} = x_{i} - \frac{f (x_{i})}{f^{'} (x_{i})}

The idea is to find a function $f (x)$ for which $x = \frac{1}{D}$ is zero. One such function is:

f (x) = \frac{1}{x} - D

(Substitue $x = \frac{1}{D}$ in the above equation and it should result in zero)

Next, we find $f^{'} (x)$ and substitue it in the NM equation to give us an equation that allows successive improvements.

x_{i + 1} = x_{i} - \frac{\frac{1}{x_{i}} - D}{- \frac{1}{x_{i}^{2}}} = x_{i} \cdot (2 - D x_{i})

Astute readers will notice that the result $x_{i + 1}$ depends on $x_{i}$ (the previous iteration) i.e. $x_{2}$ depends on $x_{1}$ and $x_{1}$ depends on $x_{0}$ How do we calculate $x_{0}$ ? This is the problem of initial approximation.

Initial Approximation to the Reciprocal

Lest the curtain be drawn too soon, here is the final equation for calculating $x_{0}$ , provided $D$ has been scaled to be in the range $[0.5, 1]$ :

x_{0} = \frac{48}{17} - \frac{32}{17} D

This equation is a linear, smooth, and non-periodic function. In numerical algorithms like division, the goal is not necessarily the smallest average error, but guaranteeing the worst-case error ( $| ϵ_{0} |$ ) is as small as possible. This predictability is important because the initial error directly determines the number of Newton-Raphson iterations required to reach full machine precision.

We wish to calculate an approximation for the function $1 / D$ such that the worst-case error is minimal. The right tool for this job is the Chebyshev Equioscillation Theorem.

Chebyshev approximation is used because it provides the Best Uniform Approximation (or Minimax Approximation). This means that out of all possible polynomials of a given degree, the Chebyshev method yields the one that minimizes the maximum absolute error across the entire target interval.

We start by formulating the error function on which equioscillation will be applied. The error function for figuring out the reciprocal $x_{0} = 1 / D$ using a simple straight line $x_{0} = T_{0} + T_{1} \cdot D$ (a linear equation) tells us how far off our guess is from the perfect answer. Because we want the total result $D \cdot x_{0}$ to be near $1$ , we make the error function $f (D)$ measure the difference between that product and $1$ . The formula is $f (D) = 1 - D \cdot x_{0}$ . When we plug in the straight-line guess, the formula becomes:

f (D) = 1 - T_{0} D - T_{1} D^{2}

The main goal is to pick the numbers $T_{0}$ and $T_{1}$ that minimize the absolute value of this error $f (D)$ everywhere in the range. This is exactly what the Chebyshev method does.

Before we apply the theorem on the error equation, we need to constrain the values that $D$ can take. Bounding $D$ guarantees that the starting error is small enough for the subsequent iterations to converge quickly and predictably to full fixed-point precision. Without this bound, a much more complex, higher-degree polynomial would be needed, defeating the efficiency goal. We bound $D$ to be $[0.5, 1]$ . In code, this scaling can be achieved through simple bit-shifts. As long as we scale the numerator too, the result remains correct.

The theorem states that a polynomial is the best uniform approximation to a continuous function over an interval if and only if the error function alternates between its maximum positive and maximum negative values at least $(n + 2)$ times, where $n$ is the degree of the polynomial. Since $n = 1$ (linear approximation), we need $1 + 2 = 3$ alternating extrema: at the two endpoints ( $D = 1 / 2$ and $D = 1$ ) and the local extremum ( $D_{m i n}$ ) between them.

The location of the local extremum is found by setting the derivative to zero:

f^{'} (D) = - T_{0} - 2 T_{1} D = 0, which gives: D_{m i n} = - \frac{T_{0}}{2 T_{1}}

The first condition of the theorem is that the error magnitude is equal at endpoints, i.e., $f (1 / 2) = f (1)$ .

1 - \frac{T_{0}}{2} - \frac{T_{1}}{4} = 1 - T_{0} - T_{1}

This simplifies to:

T_{0} = - \frac{3}{2} T_{1}

The second condition states that the error at endpoints must be the negative of the error at the extremum, i.e., $f (1) = - f (D_{m i n})$ .

1 - T_{0} - T_{1} = - (1 - T_{0} D_{m i n} - T_{1} D_{m i n}^{2})

Substituting $D_{m i n}$ and simplifying:

1 - T_{0} - T_{1} = - (1 + \frac{T_{0}^{2}}{4 T_{1}})

Substituting the value of $T_{0}$ from above:

1 - (- \frac{3}{2} T_{1}) - T_{1} = - 1 - \frac{(- 3 / 2 \cdot T_{1})^{2}}{4 T_{1}} 1 + \frac{1}{2} T_{1} = - 1 - \frac{9 T_{1}}{16}

Solving this linear equation for $T_{1}$ :

2 = - \frac{17 T_{1}}{16} T_{1} = - \frac{32}{17}

Substituting $T_{1}$ back into the original equation to find $T_{0}$ :

T_{0} = - \frac{3}{2} \cdot (- \frac{32}{17}) T_{0} = \frac{48}{17}

The resulting linear approximation is $x_{0} = T_{0} + T_{1} D$ :

x_{0} = \frac{48}{17} - \frac{32}{17} D

This equation gives the optimal initial estimate for the reciprocal that can be refined by iterations of the Newton-Raphson equation.

Implementation

(For convenience, here’s a link to the complete implementation)

The original rationale for this implementation was two-fold:

Lack of division circuitry.
Lack of support for floating point types.

Lack of division circuitry is solved by the algorithm design (reciprocal multiplication). Lack of floating point types is dealt with by using Fixed-Point notation. Fixed point allows storing everything in integers and operating using bit-shift operations, which are highly cost-efficient on embedded hardware.

Here’s an implementation of a fixed-point type in C++:

#include <iostream>
#include <cmath>
#include <cstdint>
#include <type_traits>
#include <limits>
#include <cassert>
#include <numeric>

template <int FRAC_BITS, int TOTAL_BITS>
class FixedPoint {
    using BaseType = decltype([] {
        if constexpr (TOTAL_BITS <= 8) {
            return int8_t{};
        } else if constexpr (TOTAL_BITS <= 16) {
            return int16_t{};
        } else if constexpr (TOTAL_BITS <= 32) {
            return int32_t{};
        } else {
            return int64_t{};
        }
    }());

    using WideType = typename std::conditional<(sizeof(BaseType) <= 4), int64_t, __int128_t>::type;
    BaseType value;

public:
    constexpr static BaseType SCALE = static_cast<BaseType>(1ULL << FRAC_BITS);
    FixedPoint() : value(0) {}
    FixedPoint(float f) : value(static_cast<BaseType>(std::round(f * SCALE))) {}
    BaseType raw() const { return value; }

    static FixedPoint fromRaw(BaseType rawVal) {
        FixedPoint fp;
        fp.value = rawVal;
        return fp;
    }

    float toFloat() const { return static_cast<float>(value) / SCALE; }

    FixedPoint operator+(const FixedPoint &other) const {
        return fromRaw(value + other.value);
    }
    FixedPoint operator-(const FixedPoint &other) const {
        return fromRaw(value - other.value);
    }
    FixedPoint operator*(const FixedPoint &other) const {
        WideType prod = static_cast<WideType>(value) * other.value;
        return fromRaw(static_cast<BaseType>(prod >> FRAC_BITS));
    }
    FixedPoint operator*(float factor) const {
        return FixedPoint(toFloat() * factor);
    }

    friend std::ostream &operator<<(std::ostream &os, const FixedPoint &fp) {
        return os << fp.toFloat();
    }
};

The template parameters specify the fractional and total bit lengths. For example, FixedPoint<8, 16> uses a 16-bit integer with the scale $2^{8}$ . The central idea is that all operations are performed on the integer directly, and conversion involves scaling and de-scaling with the constant scale value.

The division function below implements the 4-part process: Normalization, Initial Approximation, NR Iterations, and Multiplication with Numerator. We use the higher precision FixedPoint<16, 32> (Fx16_32) for intermediate calculations to minimize approximation error before truncating to the final Fx8_16 result.

using Fx8_16 = FixedPoint<8, 16>;
using Fx16_32 = FixedPoint<16, 32>;

Fx8_16 fxdiv_corrected(int n, int d) {
    assert(d != 0 && "Divide by zero undefined");

    if (n == 0) return Fx8_16(0.0f);

    bool result_is_negative = ((n < 0) != (d < 0));
    uint32_t nv = static_cast<uint32_t>(std::abs(n));
    uint32_t dv = static_cast<uint32_t>(std::abs(d));

    // 1. Normalization/scaling 'd' to fit between 0.5 and 1.0
    int shift = std::countl_zero<uint32_t>(dv);
    uint32_t scaled_val_raw = dv << shift;
    uint32_t scaled_val_n_raw = nv << shift;
    constexpr int INTERPRETATION_SHIFT = 31 - 16;

    Fx16_32 d_scaled = Fx16_32::fromRaw(static_cast<int32_t>(scaled_val_raw >> INTERPRETATION_SHIFT));
    Fx16_32 n_scaled = Fx16_32::fromRaw(static_cast<int32_t>(scaled_val_n_raw >> INTERPRETATION_SHIFT));

    // 2. Initial approximate calculation (Chebyshev: X0 = 48/17 - 32/17 * D)
    Fx16_32 a {2.8235294f}; // 48/17
    Fx16_32 b {1.8823529f}; // 32/17
    Fx16_32 initial_approx = a - (b * d_scaled);

    // 3. Newton-Raphson iterations
    Fx16_32 val = initial_approx;
    val = val * (Fx16_32{2.f} - (d_scaled * val)); // E1: Precision ~8 bits
    val = val * (Fx16_32{2.f} - (d_scaled * val)); // E2: Precision ~16 bits (sufficient for Fx16_32)
    val = val * (Fx16_32{2.f} - (d_scaled * val)); // E3: Conservative overkill
    val = val * (Fx16_32{2.f} - (d_scaled * val)); // E4: Conservative overkill

    // 4. Multiplication with Numerator
    Fx16_32 res_16_32 = n_scaled * val;

    if (result_is_negative) {
        // Simple negation
        res_16_32 = res_16_32 * -1.0f;
    }

    // Truncate from Fx16_32 (FRAC_BITS=16) to Fx8_16 (FRAC_BITS=8)
    constexpr int TRUNCATION_SHIFT = 16 - 8;

    // Shift the raw 32-bit value right by 8 bits to change the binary point position
    int32_t raw_final_32 = res_16_32.raw();
    int16_t raw_final_16 = static_cast<int16_t>(raw_final_32 >> TRUNCATION_SHIFT);

    Fx8_16 final_res = Fx8_16::fromRaw(raw_final_16);

    return final_res;
}

The INTERPRETATION_SHIFT and TRUNCATION_SHIFT variables account for correctly aligning the binary point.

The INTERPRETATION_SHIFT (31 - 16 = 15) adjusts the $Q 1.31$ normalized input to the $Q 15.16$ format used for calculation, ensuring the value is interpreted as being in the range $[0.5, 1.0]$ .
The TRUNCATION_SHIFT (16 - 8 = 8) reduces the $Q 15.16$ result to the final $Q 7.8$ output format, discarding the lower 8 fractional bits.

The code is compiled and run with an example:

int main() {
    std::cout.precision(10);
    int n = 3;
    int d = 4;
    Fx8_16 result = fxdiv_corrected(n, d);
    std::cout << "Approximate division of " << n << "/" << d << ": " << result << std::endl;
    std::cout << "Real division of " << n << "/" << d << ": " << static_cast<float>(3)/static_cast<float>(4) << std::endl;
    return 0;
}

$ clang++ fixed_div.cpp -o a -std=c++20
$ ./a
Approximate division of 3/4: 0.74609375
Real division of 3/4: 0.75

The result $0.74609375$ is produced. The error is $| 0.75 - 0.74609375 | = 0.00390625$ . This error is precisely $2^{- 8}$ , confirming that the final precision is limited by the $Q 7.8$ output format’s smallest bit, which is a key design feature of fixed-point systems. In fact, the number $0.00390625$ has a name: ULP. Due to our use of $Q 7.8$ as the resultant type, it is impossible to precisely express $0.75$ . What we can express is $0.75 - U L P$ and $0.75 + U L P$

Optimizations

Power-of-Two Denominator Shortcut

When the denominator $D$ is a power of two ( $\pm 2^{k}$ ), the division $N / D$ simplifies to a bit shift. This avoids the computationally expensive Newton-Raphson (NR) iterative loop entirely. The check to detect if an integer is a power of 2 can be performed in $𝒪 (1)$ time, providing a major speed increase for such common cases.

Exploiting Quadratic Convergence

One property of NR iteration is Quadratic Convergence. Every iteration approximately doubles the number of correct bits in the result.

The optimization involves determining the required number of iterations based on the fractional length of the resultant type. For a required precision of $16$ fractional bits (as in the Fx16_32 intermediate type), only two iterations are mathematically necessary after the initial approximation. constexpr if statements can be used to compile-time check the required precision and eliminate unnecessary NR iterations, ensuring no runtime performance penalty.

Alternative Initial Approximation Methods

While the Chebyshev approximation provides the optimal minimax error for the initial guess, alternative methods exist:

Remez Algorithm: Can generate even better approximations using a higher-degree polynomial. The trade-off is higher initial calculation complexity for the potential benefit of requiring fewer subsequent NR iterations.
Look-up Tables (LUTs) and Magic Constants: Recent research shows that pre-computed LUTs combined with constants can speed up the initial reciprocal approximation.
CORDIC (Coordinate Rotation Digital Computer): An alternative iterative technique that can calculate division and other trigonometric functions using only additions and shifts. It offers competitive performance in some hardware environments.

Conclusion

This article came into being after I spent around a month trying to understand approximation methods and getting code for it to work. The actual code was contributed to the llvm-project as an addition to the stdfix library in the llvm libc project. Here is the PR. Here’s a link to the complete division function implemented with stdfix primitives and the optimizations mentioned above.

Typo Correction in LLVM

2025-09-07T00:00:00+05:30

If you’re in the business of writing code, you might have noticed that your compiler is capable of identifying and suggesting fixes for the typos in your programs.

For example, take the following code:

int foo(int x, int y) {
  int pacman = x * y;
  return pamcan;
}

Compiling this generates an error identifying that pamcan is an undeclared identifier and a valid identifier by the name pacman is available

$ clang -c a.c -o a.o
a.c:3:10: error: use of undeclared identifier 'pamcan'; did you mean 'pacman'?
    3 |   return pamcan;
      |          ^~~~~~
      |          pacman
a.c:2:7: note: 'pacman' declared here
    2 |   int pacman = x * y;
      |       ^
1 error generated.

We are interested in how the compiler figures out that pacman is a valid correction for our typo. This article explains how the compiler (clang, to be specific) does it.

Parsing and Semantic Analysis

This type of error is detected by the compiler’s Semantic Analyzer, a component of the parser. In clang, the semantic analyzer is known as Sema

Grepping for the diagnostic message “use of undeclared identifier” within the llvm-project monorepo leads to the file DiagnosticSemaKinds.td.

~/dev/llvm-project/clang $ rg "use of undeclared identifier" -g '!*test*' -g '!*docs*' -g '!*www*'
include/clang/Basic/DiagnosticSemaKinds.td
6111:def err_undeclared_var_use : Error<"use of undeclared identifier %0">;
6113:  "use of undeclared identifier %0; "
11285:  "use of undeclared identifier %0; did you mean %1?">;

This is a TableGen file, an abstraction used by LLVM to maintain information files. It is compiled into the DiagnosticSemaKinds.inc header file. The specific diagnostic is declared as the following and can be found in the build directory:

DIAG(err_undeclared_var_use_suggest, CLASS_ERROR, (unsigned)diag::Severity::Error, "use of undeclared identifier %0; did you mean %1?", 0, SFINAE_SubstitutionFailure, false, true, true, false, 3)

Now, we can begin our detective hunt for the part of code we’re interested in. We’ll use the trustworthy ‘grep’ again to search for err_undeclared_var_use.

We land on Sema::DiagnoseEmptyLookup, which uses the string we searched for. As the name suggests, this function figures out what to do when a symbol is not present in the symbol table. It first tries to check if this is an unqualified look up. If this fails, it tries to correct for a typo. This is the snippet where the decision is made:

  // We didn't find anything, so try to correct for a typo.
  TypoCorrection Corrected;
  if (S && (Corrected =
                CorrectTypo(R.getLookupNameInfo(), R.getLookupKind(), S, &SS,
                            CCC, CorrectTypoKind::ErrorRecovery, LookupCtx))) {
    std::string CorrectedStr(Corrected.getAsString(getLangOpts()));
    ...

It calls the function Sema::CorrectTypo which mainly sets thresholds for what results are acceptable as corrections or fails otherwise. CorrectTypo calls Sema::makeTypoCorrectionConsumer. makeTypoCorrectionConsumer iterates over available identifiers, calls FoundName which adds a name if its edit distance is less than a particular threshold. We ultimately land on the ComputeMappedEditDistance function, which is the meat and potato of this operation.

Following text will be discussing this.

Levenshtein Distance

The Levenshtein Distance, or edit distance, quantifies the minimum number of single-character edits (insertions, deletions, or substitutions) required to change one string into another. LLVM implements a space-optimized, dynamic programming solution for this in the ComputeMappedEditDistance function.

Here’s an abridged but pure C++ implementation of the distance function implemented by LLVM:

size_t edit_distance(const std::string &s1, const std::string &s2) {
  size_t m = s1.size();
  size_t n = s2.size();
  std::vector<size_t> row(n + 1);
  for (size_t i = 1; i < row.size(); ++i) {
    row.at(i) = i;
  }

  for (size_t y = 1; y <= m; ++y) {
    row.at(0) = y;
    size_t best_this_row = row.at(0);
    size_t previous = y - 1;
    auto cur_item = s1.at(y - 1);
    for (size_t x = 1; x <= n; ++x) {
      int old_row = row.at(x);
      row.at(x) = std::min(previous + ((cur_item == s2.at(x - 1)) ? 0u : 1u), std::min(row.at(x-1), row.at(x) + 1));
      previous = old_row;
      best_this_row = std::min(best_this_row, row.at(x));
    }
  }
  size_t res = row[n];
  return res;
}

The algorithm can be understood by visualizing a 2D grid, D, where D[i][j] represents the Levenshtein distance between the first i characters of the first string and the first j characters of the second. The algorithm fills this table, and the final distance is the value in the bottom-right cell, D[m][n].

The code implements this by translating the recursive definition into an iterative process. It maintains only the current and previous rows of the conceptual table to save space.

Base Cases: The costs for transforming an empty string into a prefix of another string are represented by the first row and column of the table. In the code, this is handled by the initial loop that populates the row vector and the line row.at(0) = y; within the main loop.
Recursive Step: The calculation for D[i][j] is based on three smaller subproblems: deletion, insertion, and substitution. The code calculates the value for row.at(x) (which corresponds to D[i][j]) by using values from the previous row (previous and old_row) and the current row (row.at(x-1)). This mirrors the recursive formula:

1 + min({deletion}, {insertion}, {substitution}).

Experiments with the Typo Correction System

Armed with the knowledge that suggestion for typo correction is based on distance between two words, there should be a threshold after which suggestions should be discarded. We can find the code for this in Sema::CorrectTypo:

  // Make sure the best edit distance (prior to adding any namespace qualifiers)
  // is not more that about a third of the length of the typo's identifier.
  unsigned ED = Consumer->getBestEditDistance(true);
  unsigned TypoLen = Typo->getName().size();
  if (ED > 0 && TypoLen / ED < 3)
    return FailedCorrection(Typo, TypoName.getLoc(), RecordFailure);

  TypoCorrection BestTC = Consumer->getNextCorrection();
  TypoCorrection SecondBestTC = Consumer->getNextCorrection();
  if (!BestTC)
    return FailedCorrection(Typo, TypoName.getLoc(), RecordFailure);

If the best edit distance “is not more than a third of the length of the typo’s identifier”, it’ll move to the next correction. If there are no other corrections, it exits early.

We can, in fact, trigger this by changing the variable name so that it’s more than 1/3 of typo’s length. We need atleast TypoLen of 6 and an edit distance ED of 2.

Here’s one example,

int foo(int a) {
  int pacman = a + 1;
  return pamcaa;
}

Compiling this results in,

~/dev/llvm-project/build-clang $ ./bin/clang-22 -c a.c -o a -target riscv64
a.c:3:10: error: use of undeclared identifier 'pamcaa'
    3 |   return pamcaa;
      |          ^~~~~~
1 error generated.

No suggestions for this one! As expected.

Why Even Bother With FPGAs?

2024-12-22T00:00:00+05:30

Why Even Bother With FPGAs?

FPGAs being alternative processors enjoy a fair bit of skepticism, especially from people higher up in the pyramid of computer abstractions (Software Engineers and the like). This post is my attempt at trying to persuade the skeptics by way of an instance where FPGAs blow every other kind of processor out of the water.

TLDR; FPGAs can allow full DNN inference at nanosecond latency only limited by the time it takes for electrons to move across a circuit. In comparison, CPU/GPUs may only be able to run a couple instructions in nanosecond timeframe, entire inference will require many million/billion of these instructions.

FPGAs for the Unenlightened

FPGAs are circuit emulators. Digital Circuits consists of logic gates and connections between them, FPGAs emulate logic gates and their connections.

Logic gates can be represented by their Truth Table. Truth tables are a form of hash table where the key is a tuple of binary values corresponding to each input and output is a single bit representing the output of the gate. One kind of FPGA (SRAM-based), emulate logic gates by storing truth tables in memory.

Connections are emulated via Programmable Interconnects. Think of a network switch, programmable interconnects are pretty much like the same except on a very low-level. This document explains in detail the different VLSI architectures present in modern FPGAs.

A programmer usually does not describe circuits in the form of logic gates, they use abstractions in the form of HDLs to behaviorally describe operations that a circuit must perform. A compiler converts/maps HDL programs to FPGA primitives.

As it should be obvious by now, FPGAs are unlike processors. They do not have any "Instruction Set Architecture". If there is a need, the programmer must design and implement an ISA[^1]. FPGAs require thinking of problems as circuits with inputs and outputs.

The Central Argument for FPGAs

Now, let's build the argument.

Deep Neural Networks (DNN) inference on demands a lot of compute and is a pretty challenging problem. Solutions to this problem manifests in the form of ASIC accelerators and GPUs. More performance can always be brought by scaling said processors but of-course there is a limit to how far one can scale. For example, on the NVIDIA Jetson Nano the time taken to infer a single image for the CNN model ResNet50 is \~72ms. What if we needed something much faster, say the same inference in integral nanoseconds? GPUs/ASICs would only be able to execute a couple instructions in that timeframe let alone complete the inference. Certainly they won't suffice.

This requirement is not made up. Nanosecond DNN inference is a real problem faced by a team at CERN working on the Large Hadron Collider.

Here's a little description of the problem from their paper:

The hardware triggering system in a particle detector at the CERN LHC is one of the most extreme environments one can imagine deploying DNNs. Latency is restricted to O(1)µs, governed by the frequency of particle collisions and the amount of on-detector buffers. The system consists of a limited amount of FPGA resources, all of which are located in underground caverns 50-100 meters below the ground surface, working on thousands of different tasks in parallel. Due to the high number of tasks being performed, limited cooling capabilities, limited space in the cavern, and the limited number of processors, algorithms must be kept as resource-economic as possible. In order to minimize the latency and maximize the precision of tasks that can be performed in the hardware trigger, ML solutions are being explored as fast approximations of the algorithms currently in use.

Solutions

There are, broadly speaking, two ways of solving this problem:

1. The ASIC Way

This includes CPUs/GPUs/TPUs or any other ASIC. The idea would be to to have a large grid of multipliers and adders to carry out as many multiply-accumulate operations in parallel. To achieve more performance, research would be put to increase the frequency of the chip (Moore's law). Compilers and specialized frameworks help abstract computation. And if, we need more performance, specialized engineers (who have mastered assembly language) are called upon to write performant kernels, making use of clever tricks to have the fastest possible dot product.

2. The FPGA way

Through this way, the idea is to exploit FPGA's programming model. Instead of writing a program for our problem, we design a circuit for it. Each layer of a neural network would be represented by a circuit. Inside the layer, all dot-products themselves are represented by a circuit. If the neural network is not prohibitively large, we can even fit the entire NN as a combinational circuit.

As you might have learnt in your digital circuits course, combinational circuits do not contain any clocks i.e. there's no notion of frequency — inputs come in, outputs go out. The speed of computation is only bottleneck'ed by the time it takes electrons to pass in that chip. How cool is that?!

Flaws with the FPGA way

One of the biggest flaw with fitting entire problems on the FPGA is that of combinatorial explosion in complexity. For example, in order to design a circuit for a multiplier, there are well known algorithms that result in very efficient multiplier. One can avoid going this route by directly encoding the multipliers into truth-tables. Instead of calculating the outputs of a multiplication, we remember and look-it-up. Here's verilog for a 2-bit multiplication:

module mul ( input signed [1:0] a, input signed [1:0] b, output signed [3:0] out);
 assign out[3] = (a[0] & a[1] & b[0] & b[1]);
 assign out[2] = (~a[0] & a[1] & b[1]) | (a[1] & ~b[0] & b[1]);
 assign out[1] = (~a[0] & a[1] & b[0]) | (a[0] & ~b[0] & b[1]) | (a[0] & ~a[1] & b[1]);
 assign out[0] = (a[0] & b[0]);
endmodule

Each output is just a combination of its inputs.

Here's the problem: this method of designing multipliers does not scale! The 2bit multiplier takes 4 LUTs (pretty reasonable). But the same for an 8bit multiplier takes \~18,000 LUTs and 3+ hrs to synthesize (awful). The increase is at the rate of 2\^n. Many large neural networks will have a hard time to fit on the FPGA in this way.

This doesn’t signal the end for FPGAs, however. There’s still a strong case to be made for their use—just as the team at CERN has demonstrated. In fact, they are actively leveraging this potential. They discovered that neural network layers can be heterogeneously quantized — meaning each layer can have a different precision level depending on its significance in the computation pipeline, as outlined in their work here

If an entire network cannot fit on an FPGA, fast reconfiguration can provide a solution. This involves configuring the hardware for one layer, processing its outputs, then reconfiguring the hardware for the next layer, and so on. The approach can be further refined to enable reconfiguration at a per-channel level, allowing smaller FPGAs with limited resources to participate. A 'compiler' would orchestrate the computation offline, determining the sequence and timing of reconfigurations before the actual computation begins.

Recent interest in hyper-quantization i.e. 1bit, 2bit, 3bit ... networks is a big win for the FPGA way. The lower the resolution, the more efficient and practical the solution becomes, making FPGAs a great fit for this approach.

Conclusion

With the FPGA way, many problems spanning different domains can be solved in interesting and (sometimes) superior ways. At my workplace, we've started research in the FPGA way, trying to bring it out of the depths of complexities and solve practical problems.

The intention of this post is not to compare ASICs and FPGAs (comparisons are futile), but to highlight how FPGAs ought to be seen and used. In the following few months, i'll write more on this research as I uncover it myself. I'll leave you with some links advocating for the FPGA way[^2]

Footnotes

[^1]: The term "architecture" is a bit overloaded. The first meaning is of the VLSI sense i.e. how LUTs and interconnect are organized to make the FPGA. Another usage is for describing what all higher level components are being designed on top of the FPGA. Think matmul engines, caches etc. "Architecture" has meaning on different levels of circuit design.

[^2]: The is a term i've coined myself. I've not seen anyone else use it in their works.

No-ISA is the Best ISA

2024-10-04T00:00:00+05:30

No-ISA is the Best ISA

This week, me and my colleague were present at the first compilertech.org workshop talking about the work we are doing at Vicharak involving FPGAs, Reconfigurable Computing and Compilers for such computers. This small blog post is a brief summary of the talk.

The slides (and the extended slides) for the presentation are available at: github.com/vicharak-in/noisa. Video for the talk will soon be available.

Summary

The talk is divided into four chapters:

Chapter 1

Chapter 1 lists the problems with modern compute, key problems being the slowdown and end of Moore's law and Dennard scaling and the von-neumann bottleneck. We ask ourselves whether compute should be restricted to a small selection of available processors/architectures. Last slide in chapter 1 lists some concrete problems where using existing compute is difficult.

Chapter 2

Chapters 2 and 3 include an introduction to reconfigurable/heterogeneous computing and EDA compilers.

Reconfiguration and Heterogeneity are the two key ideas of the architecture that we propose. A separation from von-neumann architectures, by the way of flow-based reconfigurable computers is discussed. The central theme of the idea is to make it easy/automate the generation of hardware for our algorithms instead of programs. In essence, the idea is to have a unique and optimal hardware for every software.

Chapter 3

Since solving problems through reconfigurable/heterogeneous require generation of hardware, EDA compilers and their efficiency has to be considered too. Chapter 3 is about EDA compilers being a nightmare to deal with in terms of flexibility, hackability, performance and adaptability.

Chapter 4

The last chapter is on the work done so far. For this, we've designed our own hardware (Vaaman) on which applications utilizing the Reconfigurable paradigm will be designed. Two applications on which we are actively working are: Gati (CNN accelerator) and Periplex (Peripheral Generator).

Gati is a CNN accelerator that can generate custom (optimal) accelerator hardware for every NN model.

Periplex provides easy generation and multiplexing of peripheral (UART, I2C, CAN, SPI etc.) along with linux device drivers for accessing them through POSIX APIs.

Ghidra Decompiler - CLI guide

2024-08-16T00:00:00+05:30

Ghidra Decompiler - CLI guide

Ghidra has a decompiler that unlike the rest of the program (written in java) is written in C++. This caught my attention so I started to hack on it. Unfortunately, there isn't much written on the decompiler if one wants to use it standalone, in the terminal without the ghidra GUI. This article tries to fill that void.

Building The Decompiler

Fetch and unzip the ghidra package from their github release page

$ unzip ghidra_11.1.2_PUBLIC_20240709.zip

cd into the decompiler directory and build it

$ cd ghidra_11.1.2_PUBLIC/Ghidra/Features/Decompiler/src/decompile/cpp
$ make decomp_opt -j $(nproc --all)

You should end up with a executable called decomp_opt.

Running the Decompiler

While inside the directory, export the SLEIGHHOME env variable so our decompiler can find it, then run the executable.

$ export SLEIGHHOME=/home/shreeyash/ghidra_11.1.2_PUBLIC
$ ./decomp_opt
[decomp]>

The compiler is running now waiting for commands.

Note:

Remember to always export the environment variable before running decomp_opt. You could consider tossing the two commands into a script, making life easier for you.

Decompile and view an ELF executable

Let's start with a trivial c++ program with some control flow, compile it into an executable (ELF) and decompile it.

Here's the program, save and compile it:

$ cat a.cpp
#include <iostream>
#define THRESHOLD 20
int foo() {
  return 10;
}
int main() {
  int b = foo();
  std::cout << "The threshold is " << THRESHOLD << '\n';
  std::cout << "You returned " << b << '\n';
  if (b < THRESHOLD) {
    std::cout << "get in\n";
  } else {
    std::cout << "get out!\n";
  }
}
$ g++ -no-pie a.cpp -o a
$ ./a
The threshold is 20
You returned 10
get in

The executable is ready, what's left now is decompilation.

Let's start the decompiler, and load our file:

$ ./decomp_opt
[decomp]> load file a                        
a successfully loaded: Intel/AMD 64-bit x86

We've loaded our executable in the decompiler. c++ is an abstract language with constructs that do not make any sense to a CPU. These include, but are not limited to: functions, structs, loops etc. In order to implement these, the compiler has to translate abstractions into concrete implementation which manifests itself in the form of control flow instructions like branch, compare, and jump. If we peep into an executable, we'll notice what we called functions are now 'addresses' i.e. a number that represents a location in memory. Functions are run by jumping (i.e. setting the program counter) to an address. Essentially, if we wish to decompile a function we had in source, we'll have to find the corresponding address at which it resides. a.cpp has two functions: main and foo. To find the address where a functions resides in the executable, we could use objdump.

$ objdump -C -D a
...
00000000004011c5 <main>:
4011c5:       f3 0f 1e fa             endbr64
4011c9:       55                      push   %rbp
4011ca:       48 89 e5                mov    %rsp,%rbp
4011cd:       48 83 ec 10             sub    $0x10,%rsp
4011d1:       e8 e0 ff ff ff          call   4011b6 <_Z5todayv>
4011d6:       89 45 fc                mov    %eax,-0x4(%rbp)
4011d9:       48 8d 05 24 0e 00 00    lea    0xe24(%rip),%rax        # 402004 <_IO_stdin_used+0x4>
4011e0:       48 89 c6                mov    %rax,%rsi
4011e3:       48 8d 05 96 2e 00 00    lea    0x2e96(%rip),%rax        # 404080 <_ZSt4cout@GLIBCXX_3.4>
4011ea:       48 89 c7                mov    %rax,%rdi
4011ed:       e8 9e fe ff ff          call   401090 <_ZStlsISt11char_traitsIcEERSt13basic_ostreamIcT_ES5_PKc@plt>
4011f2:       48 89 c2                mov    %rax,%rdx
4011f5:       8b 45 fc                mov    -0x4(%rbp),%eax
...

Searching for 'main' reveals its label which resides at address 0x4011c5.

[decomp]> load addr 0x4011c5 main                  
Function main: 0x004011c5

load addr takes an address and an optional 'label'. Label is essentially a name that we assign to that address. In this case, it was 'main'—could've been anything for what its worth.

[decomp]> decompile                             
Decompiling main                                   
Decompilation complete                          
[decomp]> print C                               

xunknown8 main(void)

{
  int4 iVar1;
  xunknown8 xVar2;

  iVar1 = func_0x004011b6();
  xVar2 = func_0x00401090(0x404080,0x402004);
  xVar2 = func_0x004010c0(xVar2,0x14);
  func_0x004010a0(xVar2,10);
  xVar2 = func_0x00401090(0x404080,0x402016);
  xVar2 = func_0x004010c0(xVar2,iVar1);
  func_0x004010a0(xVar2,10);
  if (iVar1 < 0x14) {
    func_0x00401090(0x404080,0x402024);
  }
  else {
    func_0x00401090(0x404080,0x40202c);
  }
  return 0;
}
[decomp]>

Just like that, we've decompiled our program. Notice how the names are garbled. This is because names (of variables and functions) are really neccessary to execute a program.

Let's analyze the decompiled output. The latter part of all function names are their address. This means, we can look them up in the objdump. Moreover, if the set of commands that got us main s decompilation we to be repeated for all the functions present in in the output, the resulting decompilation of main would replace all address with the labels we assign to them. Looking up in objdump, we find func_0x004011b6 to be foo:

...
00000000004011b6 <foo()>:
4011b6:       f3 0f 1e fa             endbr64
4011ba:       55                      push   %rbp
4011bb:       48 89 e5                mov    %rsp,%rbp
4011be:       b8 0a 00 00 00          mov    $0xa,%eax
...

func_0x00401090 is not present in the executable, however, the calls to this function are shown in the objdump thusly:

4011ed:       e8 9e fe ff ff          call   401090 <std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*)@plt>

Its quite obvious from the hint that func_0x00401090 is the operator \<\< overloaded to accept a std::basic_ostream object and a const char \*. The \@plt at the end indicates that this function can be found in the .plt section of the executable. .plt which stands for Procedure Linkage Table is a redirection table of external functions that can be found in shared objects. So, func_0x00401090 is operator\<\< found in libstdc++.so that the program is linked to. It takes two arguments: both addresses to objects. A search reveals that the first argumnet is the object std::cout of which the definition resides in an external library (libstdc++.so) and the other argument is a char literal that can be found in the .rodata section of the executable.

$ objdup -s -j .rodata a
Contents of section .rodata:
402000 01000200 54686520 74687265 73686f6c  ....The threshol
402010 64206973 2000596f 75207265 7475726e  d is .You return
402020 65642000 67657420 696e0a00 67657420  ed .get in..get
402030 6f757421 0a00                        out!..

Indeed, the string \"The threshold is \" is present at address 0x0402004.

Likewise, all following functions till func_0x004010a0 are overloads of operator\<\< that handle different types of data. What remains is the control flow. It checks if iVar1 which is b in the original source is less than 0x14 (THRESHOLD) and calls the familiar func_0x00401090 i.e. (operator\<\<).

Conclusion

Our work was made much easier by the fact that the executable was not 'stripped'. Stripping is a process that gets rid of all the symbols that are not absolutely neccessary for execution (greatly reduces executable size). In the real world, especially if we are dealing with propreitary software, executables might be stripped. Unstripped executables allows us to tread faster by simply searching for symbols like we did to find main. Stripped executables require us to trace, find and deduce what we need. In a later article, I may demo decompilation of stripped executables.

When Reverse Engineering, Your Pattern Seeking Brain Is Your Friend

2024-07-12T00:00:00+05:30

When Reverse Engineering, Your Pattern Seeking Brain Is Your Friend

At work, I've been working on reverse engineering a propreitary file format that is used to represent a synthesized netlist for FPGAs by our vendor's EDA tools.

It's a binary file, and here's a sample of the hexdump:

00000000  12 6c 1f 03 0b 00 00 a0  00 00 00 00 40 e4 7a 1f  |.l..........@.z.|
00000010  03 08 b6 84 a3 66 00 00  00 00 01 00 20 20 00 02  |.....f......  ..|
00000020  40 31 00 ab 00 0e 43 45  4e 4e 41 48 45 68 65 46  |@1....CENNAHEheF|
00000030  62 66 4b 49 03 8f 8d a3  03 8f 8d a3 02 00 08 aa  |bfKI............|
00000040  ba b9 9e 40 b6 9b 99 00  00 00 00 00 00 00 00 01  |...@............|
00000050  0a 49 4d 41 4f 40 4e 4e  49 48 82 08 aa ba b9 9e  |.IMAO@NNIH......|
00000060  40 b6 9b 99 00 40 31 00  ab 00 00 b6 84 a3 66 00  |@....@1.......f.|
00000070  00 00 00 ff 00 00 00 00  00 00 00 00 03 8f 8d a3  |................|
00000080  00 00 00 00 00 00 00 00  00 00 40 31 00 ab 00 00  |..........@1....|
00000090  b6 84 a3 66 00 00 00 00  ff 06 00 14 00 05 00 01  |...f............|

Without any information on the file, this stands as a wall full of random bytes. Although complex, there's a lot that can be deduced by looking for patterns. File formats are often divided in sections. The bytes may look random, but in reality, they ought to be very structured. The first step in dealing with this is to extract the structure.

One trick I use is to zoom out on the hexdump. This isolates zeros and all other bytes.

Here's an image of a hexdump of the same file, zoomed out:

Do you notice any patterns?

There are alternating strips of dark and light patterns. The light patterns are just zeros and darker ones appear to be 'data'. Here's a highlighted image with the patterns. White rectangles represent dark parts and greens represent the zeros.

Since this repeats it's likely encoding the same 'type' of information. The pattern starts with dark section followed by white and ends with a white section. They always come in a pair. So we can deduce that to represent one of this type of data, we need a dark part followed by the light part.

Now, what could this be?

This is a question that falls into the 'content' part. What we did above was the 'structure' part. As it turns out, getting meaning out of this is much more tedious.

A Fuzzer is the right tool for this job. As fuzzers tend to be very special purpose, i wrote one for myself. Extracting the details with the fuzzer vindicates our suspicion. The dark and light parts are indeed part of the structure. The dark part is sort of a preamble to the light part. The light part is a port reference list for all the black-box module present in the netlist.

That this pattern represents black-box modules can be deduced by counting the number of times this pattern is present, and what other thing is present as many times in the original source file from which this was generated. Inspecting the source, which is just a verilog file confirms that this are indeed the black-box modules.

Conclusion

In conclusion, file formats or any other type of data that are supposed to be regular can be brute-forced by our pattern-seeking brains to reveal their structures.

PS: I'll write a full description of the fuzzer and document other details as this project progresses.

How to remove a vertex from a boost graph?

2024-05-04T00:00:00+05:30

How to remove a vertex from a boost graph?

Here’s the code to remove a vertex from a boost graph:

#include <boost/graph/adjacency_list.hpp>
#include <boost/graph/graph_traits.hpp>

using Graph = boost::adjacency_list<boost::vecS, boost::listS, boost::directedS, int>;
using Vertex = boost::graph_traits<Graph>::vertex_descriptor;

void safe_remove_vertex(Vertex v, Graph &g) {
   boost::clear_vertex(v, g);
   boost::remove_vertex(v, g);
}

Explanation

boost::clear_vertex removes all the edges coming-in or going-out of the vertex. boost::remove_vertex removes the vertex. This two step procedure is very similar to erase-remove idiom as used on std::vectors.

Note if the template parameter VertexList (second template argument to boost::adjacency_list definition) is vecS i.e. the vertices of a bgl are stored internally in a graph, calling remove_vertex on this graph invalidates all iterators to it as all the elements need to be re-arranged inside the vector. Using invalid iterators will likely cause a segfault. On the other hand, if VertexList is listS you’re safe, as no iterators are invalidated. For more information, refer to the original doc

Extended Example

For my use case I had a directed graph representing an onnx graph that i had to compile into a lower-level IR for my compiler. This translation required vertex elimination followed by patching the graph. The clear-remove pattern only removes a vertex and its edges but does not connect the parent nodes of the node under removal to its children. Ofcourse, there is no such notion of a parent or a child node in a graph. This has to be implemented by the user. The diagram below demostrates what the code following it does.

Node 3 is the one being removed, red dashed edges are the new ones after 3 is removed. Here’s the code:

#include <boost/graph/adjacency_list.hpp>
#include <boost/graph/graph_traits.hpp>

using Graph = boost::adjacency_list<boost::vecS, boost::listS,
boost::bidirectionalS, int>;
using Vertex = boost::graph_traits<Graph>::vertex_descriptor;

std::vector<Vertex> get_parents(Vertex v, Graph &g) {
  std::vector<Vertex> ret;
  auto edges = boost::in_edges(v, g);
  for (auto itr = edges.first; itr != edges.second; ++itr) {
    Vertex src_v = boost::source(*itr, g);
    ret.push_back(src_v);
  }
  return ret;
}

std::vector<Vertex> get_children(Vertex v, Graph &g) {
  std::vector<Vertex> ret;
  auto edges = boost::out_edges(v, g);
  for (auto itr = edges.first; itr != edges.second; ++itr) {
    Vertex src_v = boost::target(*itr, g);
    ret.push_back(src_v);
  }
  return ret;
}

void connect_parents_to_children(const std::vector<Vertex>& parents,
    const std::vector<Vertex>& children, Graph &g) {
  for (Vertex i: parents) {
    for (Vertex j: children) {
      std::cout << "connecting " << g[i]->name << " to " << g[j]->name << '\n';
      boost::add_edge(i, j, g);
    }
  }
}

/* remove a vertex but connect its parents to its children */
void safe_remove_vertex(Vertex v, Graph &g) {
  std::vector<Vertex> src_vertices = get_parents(v, g);
  std::vector<Vertex> dest_vertices = get_children(v, g);
  connect_parents_to_children(src_vertices, dest_vertices, g);
  boost::clear_vertex(v, g);
  boost::remove_vertex(v, g);
}

void pass(Graph graph) {
  VertexIterator vi, vi_end, next;
  std::tie(vi, vi_end) = boost::vertices(graph);

  for (next = vi; vi != vi_end; vi = next, cnt++) {
    next++;
    if (should_remove(*vi, graph)) {
      safe_remove_vertex(*vi, graph);
    }
  }
}

PS: although its supposed to be a directed graph, i’ve used bi-directional as i sometimes require backwards iteration through it.