Posts

Before we see what torch.compile does, we should first understand pytorch’s default mode and why we’d ever want to move away from it. PyTorch runs in eager mode by default. Think of it as PyTorch reading and executing your code op by op, as Python encounters each line. It’s immediate, flexible, and great for prototyping — but it pays a Python interpreter cost on every single operation. For production and deployment, we want to skip that cost. That’s where compilation comes in. ...

Today we look at matrix multiplication (matmul, as we will call in this essay). Since, the last essay was on backprop, it was only logical to think about the most fundamental math operation that lets us do the algo. That is, matmul. Also, the numbers in this essay are going to shock you. Like really. So if you think I am making this up, you should checkout my code for this essay. ...

I’m assuming you understand the basic idea of neural networks. This essay focuses purely on the backpropagation algorithm itself. What is Backpropagation? Backpropagation is an algorithm that computes how much each weight and bias should change to reduce the loss. It tells us not just whether parameters should go up or down, but by how much, based on their actual impact on the loss function. We use math to figure out that. ...

what is this about? Today we’re looking at hashing. We’ll get into the process, the data structures and some applications. Through this article, we’ll see how we can implement dictionary operations, which are mainly: insert delete search what is Hashing? Hashing is a technique of identifying an object out of a group of similar objects. Analogy for hashing: Imagine if you took your name tiwariji and ran it through a complex mathematical function that produced 7a3f9c2e. You couldn’t look at 7a3f9c2e and figure out it came from tiwariji, but every time you hash tiwariji you’d get the same result. That, is hashing. Ideally, no two different inputs should produce the same hash output. We’ll talk about that later when we discuss hash functions. ...

Data management during compiler process When compiling a piece of code, the data in the code is stored in segments. There are five types of segments: stack heap data code BSS We will focus on stack and heap in this essay. Stack Stack allocation is the process of allocating memory for local variables and function calls in call stack. Each function gets some memory in the stack to store variables in it. Since the memory is handled by the system, its faster. But the memory is less as compared to heaps. The size required is already known before execution. The compiler allocates some memory. ...

Overview When you call int x = 0; in your code, where and how is that x stored? This is the question that I wanted to know the answer of. And in this essay, we’ll look at just that. And Why should you care? As we will understand later in the article, there are issues and subtle bugs that come when programmers don’t understand their code. Hackers want to figure out ways to exploit those bugs and gain entery into people’s systems. So knowing how, why, and where of your data, in your system is not only useful, I’d say it’s necessary. ...

do you know how your code int f(){ return 42; } turns into mov eax, 42 ret and then into B8 2A 00 00 00 C3 this? Here’s the whole essay in short: source code → something happens, Intermediate code forms → something happens again, Machine code is formed. We’ll clear the ‘something’ in this article. Life would be very simple if we humans could write 1s and 0s and directly give machine it’s prefered machine code. But since we don’t have 1000 hands per person and the outputs that we’re expecting out of computers have evolved to complexities unimaginable, we need another simpler way to talk to the machines. And that is why we have different programming languages and their compilation processes. ...