Help in understanding computer science, programming and abstraction

I know why abstraction is great, but doesn't that prevent you from learning how computers work?

Certainly not. If you want to understand the abstractions at work, then study those abstractions. If you want to understand the low-level technical details of a real, physical, computer then study those details. If you want to understand both, study both. (In my opinion, a good programmer will do that.)

You seem to have got yourself stuck in a false dichotomy, as if you can only understand one abstraction level at a time (or, worse, as if only one abstraction level exists at a time). That's rather like suggesting that it is fundamentally impossible for someone to have any understanding of both physics and mathematics.

A good start would be discovering the distinction between computer science and programming.

Eventually, I thought you will become a better programmer knowing this because you'll know what's happening rather than assuming that everything is magic.

These are not contradictory things. I have no idea how to pave a road, but I know that it is not magic.

But a month ago, I came across this book called Structure and Interpretation of Computer Programs and anything in the web suggests that this is one of the best computer science book and you will get better as a programmer when reading it.

Yes, that is absolutely true.

Mainly shouldn't we avoid all abstractions and observe what really is happening at the very low-level?

Maybe once or twice, but doing that every time will prevent you from being a good programmer. You don't need to watch the electrons flow through a gate to know that it's not magic. You don't need to see the cpu translate those electrons to the bitwise representation of a number to know it's not magic. You don't need to see those bits go down the wire to know that it's not magic. There are a few dozen abstractions necessary just to put these letters alongside one another. A few hundred probably to get them to your computer from SE's servers.

Nobody knows all of them - not in depth.

This is a very common problem with beginners in programming. They want to know how things work. They think that low level is the best level, because it gives them the most power, or control, or knowledge. It does not.

Yeah, you can't treat it as magic. But you really don't need to know that stuff either. Concatenating strings by hand isn't interesting. Rolling your own sorting algorithm isn't interesting. Because any programmer can get the same result in a few order of magnitudes less time by using something written by far better programmers decades ago.

A key skill in programming is simultaneously thinking at multiple levels of abstraction. Another key skill is building abstractions; this skill uses the previous one. Low-level programming is valuable in part because it exercises and expands both these skills.

SICP models and implements interpreters, simulators for a machine model, and a compiler to that machine model (the output of which can then be run on the simulator). The machine model, while very simple, is not inherently less low-level than x86_64. Frankly, a good amount of low-level programming is dealing with arbitrary and arcane hardware/firmware interfaces which are no better than the arbitrary rules of business logic.

I know why abstraction is great, but doesn't that prevent you from learning how computers work? Am I missing something?

Go to a magic show and you'll be entertained but you won't understand how the tricks work. Read a book on magic and you'll learn how tricks work but you still won't be entertaining.

Do both. Work hard. And you might be both.

I'ved worked at high levels SOLID, bash, UML. I've worked at low levels, TASM, Arithmetic Logic Units, Analog Circuits. I can tell you, there is no level at which you can work that there isn't some magic abstracted away from you.

The key certainly is not to understand every level of abstraction at once. It's to understand one level well enough to use it right and well enough to know when it's time to move to a different one.

Any sufficiently advanced technology is indistinguishable from magic.

Arthur C Clark

No, abstractions don't prevent you from understanding how things work. Abstractions allow you to understand why (to what end) things work the way they do.

First off, let's make one thing clear: pretty much everything you've ever known is at a level of abstraction. Java is an abstraction, C++ is an abstraction, C is an abstraction, x86 is an abstraction, ones and zeroes are an abstraction, digital circuits are an abstraction, integrated circuits are an abstraction, amplifiers are an abstraction, transistors are an abstraction, circuits are an abstraction, semiconductors are an abstraction, atoms are an abstraction, electron bands are an abstraction, electrons are an abstraction, and quarks could be an abstraction too - I don't really know, I'm just making a guess based on the pattern. By the logic that low level knowledge is required to understand how something really works, if you want to understand how computers really work, you need to study physics, then electrical engineering, then computer engineering, then computer science, and essentially work your way up in terms of abstraction. (I've taken the liberty of not mentioning that you also need to study math first, to really understand physics.)

Now realistically, the days when you could make sense of computers and programming by building your way up from the lowest level details were the earliest days of computers. By now, this field has advanced too much, to the point where it can't possibly be rediscovered from scratch by a single person. There are hundreds of thousands of very qualified people specializing at every level of abstraction, working hard daily to make advances that you can't hope to understand without spending years of studying a specific portion thoroughly and committing to keeping up with the latest advancements there.

As an example, consider this Java snippet:

public void Example() { 
    Object obj = new String("...");
    // ...
}

Unless you are well-versed in stack frames, heap data structures, concurrent generational tracing GC, memory compacting, static analysis, escape analysis specifically, virtual machines, dynamic analysis, assembly language and executable space protection, you are wrong if you think you really know what happens in practice when you run something as simple as that snippet.

As another example, consider this C snippet:

void example(int i) {
    int j;
    if(i == 0) {
        j = i * 2;
        printf("Received zero, printing %d", j);
    } else {
        printf("Received non-zero, printing %d", j);
    }
}

If you show it to a beginner, they'll tell you that when the argument is non-zero, the residual contents of a memory location will be printed, because variables are actually just memory addresses behind the scenes and when you don't initialize them it's just that you don't move anything to their address and anyway there's nothing magic about them. If you show it to a non-beginner, they'll tell you that this program's behavior is undefined for non-zero values of this function's parameter and the compiler could potentially remove the conditional, treat all arguments to this function as zero, replace all of the function's call spots with calls that pass zero as the argument, set all variables that are ever passed as arguments to this function to zero, or do other paranormal stuff.

The beginner in this example took everything he/she knows into account and arrived elegantly to a completely wrong answer because (a) he/she didn't read the spec of the language (which is an abstraction on purpose, not because C programmers aren't clever enough to understand computer architecture) and (b) tried to reason about implementation details which he/she didn't fully grasp and which have evolved way beyond his/her mental model by now. This example is fictional, but it draws from everyday real-world misconceptions - the kind that sometimes lead to perilous bugs and occasionally famous security holes.

You're saying you aim to be a better programmer. Some of the best programmers I've met are the way they are because they understand abstractions and they can carry their knowledge to any language, and adapt it to any problem they need to solve, at any level they happen to be working on. Some of the worst programmers I've met are the way they are because they insist on focusing on details and trivia which they don't really understand and which a lot of the time aren't exactly up-to-date, or relevant to the problem, or applicable in the context they attempt to use them in, or have never been true in the first place.

Don't fall into the trap. After all, there is no single level of abstraction at any given point and a person isn't limited to understanding a single level of abstraction at any given moment. You can understand one, then move on to another.

Software engineering has multiple levels of detail. Your question is "what is the most rewarding, worthy, interesting level?"

It depends on your task or on what you want to be, what you care about. For big systems you should not care much about bit shifting and clock cycles. For embedded software running on a simple micro controller you will probably want to keep an eye on the ammount of memory you use and you may have to write some primitive timing routines.

Sometimes a choice made on a high abstraction level can impact performance or resource use of your system. Recognizing that and understanding why and how will help making the better choice or finding a way to make an existing system more efficient. This works both ways: knowing there is a high level utility available will keep you from inventing your own wheel in your low level domain. So having some understanding of other levels than the one you are working on may help you be more effective.

On a personal level you may want to ask yourself "do I like laying bricks or do I want to designing houses?" Or maybe make layouts of cities. Or improve the science that makes a stronger, lighter, cheaper brick?