The GNU Compiler Collection, commonly known as GCC has a set of compilers and development for most distros. It has support for primarily C and C++ and other languages as well. Our main focus is on the various optimization flags.

Note: We shall consider only `C++` for simplicity.

Levels of Optimization

There are majorly three levels of optimizations that are frequently used with increasing order of optimization level. They are:

• -O1

• -O2

• -O3

By default, -O optimization is applied where the compiler tries to reduce code size and execution time and provide much execution time. Then we can see minor/major improvements based on the code and assembly generated. Besides these, GCC also provides many other optimization flags. The entire list is available here.

Assembly?

Yes, you heard it right the C++ code written gets compiled initially by an Assembler and generates an object file `.o`. We can see the magic of these optimization flags in the generated assembly. Let’s consider the problem of calculating the sum of the first 10001 natural numbers. We can say it using the formula (n*(n+1)/2). But let’s write a naive way of calculating it using a simple for loop and see how the compiler arrives at the direct answer by activating different optimization flags.

int calc_sum()
{
    int ans = 0;
    for(int i = 0; i < 10001; ++i)
        ans += i;

    return ans;
}

When we compile it without any flag we can see a naive code translated to assembly. I'm glad to be from ECE to understand the assembly code partially 🤧

Now let’s see the magic of the GCC compiler and how it translates the code by enabling the -O1 flag.

Boom, we can see that it directly loads the end value inside a register, then subtracts one for `(end - 1)` in the for loop. Before returning, a direct calculation `50005000` without any looping.

Amazing right but this is at least not the end, we can further improve the optimization by enabling the `-O2` and `-O3` flags. For this snippet, the assembly code is the same.

🎉 Direct computation of the result just by storing the exact required value in the register. All this is not magic but the various other flags that get enabled during the activation of various optimizations.

Tradeoffs!

When we need something, we need to lose something. In this context, if we need to get accurate results, then it’s recommended practice to use the defaults. The reason is numerous flags get enabled and difficult to understand the root cause of the failure. Or else you can turn on the desired optimization level, which could have a greater compilation time but faster execution time.

You can find the code here with the generated assembly without any installation. Also, you can play with all flags by enabling the respective flags like -O3, etc, in the right panel on the placeholder named `Compiler options…`

Stay connected for more such content 🤝

Analogy to Looping Statements

In looping statements, when the execution flow enters the loop, it will be executed until it ends and the flow comes out of the loop body.

But say we need to iterate over the loop but need the flexibility to call each iteration only when required. This is where the yield statement comes into the picture. Whenever a function has a `yield` statement it has to be used inside a function and we call that a generator function that returns the generator object.

We can call the `next()` method on the generator object to yield the result. If we want to run the generator object as a loop, wrap the generator object with a list, or tuple, the rest will be handled by Python.

Features

Listed below are some of the many features and advantages of using the `yield`. A few of these are covered in the code snippet. The rest can be viewed in this video.

• It can be used for file processing

• Used in cases where the storage is limited

• Lazy sequences and collatz

• Generator comprehensions

• Generator features

• Send, throw, and close

P.S.: I have added a benchmark as well to show the difference between generic loop and yield based loop.

Stay connected for more such content 🤝

From the title, it is clear that it relates to the Walrus. The `:` represents the eyes and the `=` represents the tusks.

This is more about the use case rather than explaining through content. The real zeal comes when implementing it in real life scenarios.

About

• Representation: `:=`

• This assigns values to variables as part of functions, list comprehension, etc.

• Available from Python >= 3.8

• Used to reduce complexity and improve readability.

Use cases

• Assignment

• In-built

• User-defined

• List comprehension

Code: https://colab.research.google.com/drive/1Cm3li01GVBIQhNwWzXQdIGXn_s7Zpr-f?usp=sharing

Please stay connected for more such content🤝

File/directory permissions need to be carefully determined to provide the right level of access to the right users. In that sense, the command `chmod` comes into the picture which help us to provide appropriate access.

Starting with different user roles:

• (U) User - Manager i.e. with root user permission and complete control over the system.

• (G) Group - Team i.e. comes under the owner with certain restricted access.

• (O) Other - 3rd party i.e. everyone who isn’t part of owner/group.

Different file permissions:

• (R) Read - Just to view the contents of the script.

• (W) Write - Add content to the script.

• (X) Execute - Provide run access to the script.

Different file types:

• (-) File

• (d) Represents a directory

Note: (- `hyphen`) refers to no particular file permission for that specific group except the first position i.e. file type.

An example of a default file permission that gets created if you are a root user for a file named `temp.txt`.

`777`

If you feel allergic to alphabets then we can move to numericals. The equivalent of what is being shown in the alphabet. This table summarizes the values for all user levels and permissions.

So for example, if you want to grant complete access to a particular file then we can use the command:

chmod 777 <file.extension>

The value is derived by summing all the values which add up to `777` that’s its significance. If you want to provide just read access to the owner you can use the value `400`.

If you are still lazy then this online tool does the work job for you: https://chmod-calculator.com.

Stay connected for more such content.

What is GIL?

The Python interpreter built in CPython ensures each Python byte code uses only one thread/process at a time thus locking the interpreter from accessing free resources for faster computation. Hence the term GIL.

Why GIL?

There has been much conversation around this topic even by Guido Van Rossum, the creator of Python Programming. The primary reason for GIL is to restrict concurrent access to the CPython interpreter. The reason for the restriction is CPython interpreter gives users the freedom to run multiple interpreters in the same process. This leads to shared resources and leads to unexpected bugs in the future.

Then how to run concurrently?

Though we have such constraints we have the flexibility to run concurrently by running them as subprocesses using the `multiprocessing`. So we are bypassing multiple threads by using the strength of multiprocessors. This is independent of POSIX and Windows.

This comes with complex issues and features. One such is considering the fact there is a subprocess that’s consuming a lot of time and doesn’t complete the execution. So if we have the process as a daemon, if the main program exits then it doesn’t depend on the completion of the sub-processes. This introduces unexpected behavior in Python but we need to lose something to gain something.

Benchmark

Now that we have seen the modules needed, we would see a simple implementation of calculating the sum of first `n` natural numbers and know the difference between the execution times of various methods. Attaching the results of multiplying the first 1 lakh natural numbers with 10 threads using the basic Google colab configuration.

On varying the threads in the power of 2 we can find the optimal number of threads.

Miscellaneous

Besides the built-in features, we can induce multiprocessing in the following ways:

• ctypes: An interlink between C and Python that allows calling the DLL or `.so` files. Since C supports multi-threading it gives the freedom to run the code in multiple threads but we need to be careful and be prepared for the behavior.

• pybind: A C++ and Python interlink similar to `ctypes` but mainly to create Python bindings for existing C++ code.

• Using numpy, jit, numba, and more can speed up the execution without using multithreading.

Note:

• They both seem similar but the most difficult part is dealing with memory mapping and resource management. Eg: C++ has `int` of say 16 bits, but in Python, everything is an object. So ensuring proper resource management is the key here.`

• There has been a proposal to disable GIL but this proposal can go both ways as mentioned.

References: PEP-683, PEP-684, PEP-703

Code: https://colab.research.google.com/drive/1DWShJ8pDdVEcf5pFN5kWTGYigkwzXmWk?usp=sharing

Stay connected for more content 🤝
P.S.: Do let me know if there is any bias or mistakes. Happy to correct them going forward.

In a production environment where the data is being streamed, it is indefinite (doesn’t fit the memory), and the condition arises where we would need to get random samples of size `k` with equal probability at any point in time. This concept is called Reservoir sampling and comes in handy when the memory space is limited but at the same time, we have lots of data incoming from multiple streams like Kafka, Kinesis, etc.

To be precise equal probability (k / n) means from the reservoir of length `k` at any point of time where the length of current streamed data is `n` then for the `a[i]` element its probability is defined as:

If we keep multiplying the values all we are left with is called the equal probability.

In programming terms consider a reservoir(array) of fixed length `k`. When `n` is less than `k` then well and good.

If not then generate a random index between 1 to n. If that lies between 1 and k, then replace it with the nth element. Or else continue to the next element in the stream. The pseudocode for the same can be found here.

int arr[k];  # consider 1-based indexing

while (stream):
    if (n < k)
        # Then add to the reservoir
    else
        i = generate_random(n) # range of `i`: [1,k]
        if (i < k)
            # Then replace the reservoir element: arr[i]=nth_element
        else
            # do nothing
        fi
    fi
end loop

A problem with a similar approach can be found here.

Stay connected for more such content 🤝

When we are working with classes in Python or any modern programming language that supports OOPS, then we more often come across the terms: instance, class, and static methods. Let’s understand the differences by the end.

Now let’s use a base picture of a class named `Employee` with basic details and it has other methods with its individual use cases.

class Employee:
    num_of_employees = 0

    # Instance Method
    def __init__(self, id: int, name: str, salary: float):
        self.id = id
        self.name = name
        self.salary =salary
        Employee.num_of_employees += 1

    # Instance Method
    def display_employee_details(self) -> str:
        return f"Employee with ID: {self.id} called as {self.name}, has a salary of {self.salary}"

    @classmethod
    def count_of_employees(cls):
        print(f"Number of employees: {cls.num_of_employees}")

    @staticmethod
    def is_valid(id: int) -> bool:
        if (id < 0 or id > 99):
            return False
        else:
            return True

Let’s use this code as a base and proceed further.

P.S.: Avoid the logic for validation and print statements😅

Instance Method

Any method that has the current instance of the class as its argument then we can call that as an Instance Method. They are contained and can be used to manipulate the values for a particular instance of the class where changes in one instance don’t affect the others. They are the first argument of the method and are not passed as parameters explicitly. We can give it any name but the common naming convention is `self`.

Class Method

They are limited to the class and have access to the class itself and have access to the nearest parent class. It is wrapped using the `@classmethod` decorator in Python. When we have a class derived from the base class, then the `@classmethod` of the derived class accesses the derived class method.

If a class method is called for a derived class, the derived class object is passed as the implied first argument.

~ From the official Python docs

Static Method

They are associated with a class rather than an instance. This is useful when we need access to a function from the class without using its instance. This method is wrapped using the `@staticmethod` decorator as against using the `static` keyword in C++ and Java.

Why the @staticmethod then?

If we can access the method without instance then we can declare it globally and access it. Let’s consider the `isValidId` method is present inside the `Employee` class because it has its meaning to check if the current employee id is valid. If we have it globally and give it a more meaningful name like `is_valid_employee` without modifying the logic does the job. But the real zeal of OOPs is not consumed hence it is recommended to use the `@staticmethod` where and when applicable with appropriate tradeoffs.

Code with executable cases to understand the working: https://colab.research.google.com/drive/1gti1ksYICLurvVYEzduI2UNGptXjuh2d?usp=sharing

Stay connected for more content 🤝

Understanding the Problem

Now let’s consider the case of adding 0.1 and 0.2. The answer is 0.3. This is the output from the calculator. Pretty straightforward, but let’s jump to the twist.

Then what is so special about this lies in its actual working in most of the programming languages. For simplicity let’s consider Python. This is the output when the code is executed on a Python interpreter.

Pause for a second to analyze the root cause. Now let’s understand what could be the actual reason.

Root cause

We can conclude that the sum of 0.1 and 0.2 is not 0.3. Then what is the value of the same? So let’s execute and find it out.

What’s this weird number😵‍💫? Now let’s dig deep into what could have led to this.

In Python, floating point numbers are represented as 32 or 64-bit values based on the compiler using the IEEE-754 standard. So now let’s convert all the values into their respective binary.

From this, one can observe that the sum of 0.1 and 0.2 equate to `ans` and this is not as expected but happens internally which is the actual reason for its evaluation to be false.
The repeated mantissa values cause overflow during addition thereby leading to round-off and resulting in decimation.

Code: https://colab.research.google.com/drive/1tgHYBz0kWZ96weMa1dIuGx2Jiyo6OFGX?usp=sharing

Stay connected for more content 🤝

In the current tech world, all the content that prevails is short-span and forgotten more often. Hence there must be a solution to persist the learning that will be useful in the long run. Hence this idea of a newsletter erupted.

Why Engineering Dosage?

A simple fact is that I’m from an Engineering background and we consume information byte-sized like medicine that either provides relief from a problem (issue) or improves the current state of your body (concept). All medicines are prescribed in dosage hence the name the Engineering Dosage.

About me

I’m Sriniketh J, a passionate software and ML developer enthusiastic about the latest advancements. Always keen to learn and be curious. I always like to upskill myself. Ask me about open-source, AI-ML, software development, or anything in general. Always willing to collaborate and contribute to the community. I'm inculcating the habit of reading books. I like to play füßall a lot and learning deutsch as well.

P.S.: I had been having this idea about a year ago but it just stayed in my to-do list. This time I would make it live and be consistent.

Know more about me here: https://sriniketh.design/

Support my work: https://www.buymeacoffee.com/srini047

Cheers,

-Sriniketh J.