What is cache in C++ programming? [closed]

Question

Firstly I would like to tell that I come from a non-Computer Science background & I have been learning the C++ language. I am unable to understand what exactly is a cache? It has different meaning in different contexts. I would like to know what would be called as a cache in a C++ program? For example, if I have some int data in a file. If I read it & store in an int array, then would this mean that I have 'cached' the data? To me this seems like common sense to use the data since reading from a file is always bad than reading from RAM. But I am a little confused due to this article.

In a CPU there can be several caches, to speed up instructions in loops or to store often accessed data. These caches are small but very fast. Reading data from cache memory is much faster than reading it from RAM.

It says that reading data from cache is much faster than from RAM. I thought RAM & cache were the same. Can somebody please clear my confusion?

EDIT: I am updating the question because previously it was too broad. My confusion started with this answer. He says

RowData and m_data are specific to my implementation, but they are simply used to cache information about a row in the file

What does cache in this context mean?

Answer 1

Any modern CPU has several layers of cache that are typically named things like L1, L2, L3 or even L4. This is called a multi-level cache. The lower the number, the faster the cache will be.

It's important to remember that the CPU runs at speeds that are significantly faster than the memory subsystem. It takes the CPU a tiny eternity to wait for something to be fetched from system memory, many, many clock-cycles elapse from the time the request is made to when the data is fetched, sent over the system bus, and received by the CPU.

There's no programming construct for dealing with caches, but if your code and data can fit neatly in the L1 cache, then it will be fastest. Next is if it can fit in the L2, and so on. If your code or data cannot fit at all, then you'll be at the mercy of the system memory, which can be orders of magnitude slower.

This is why counter-intuitive things like unrolling loops, which should be faster, might end up being slower because your code becomes too large to fit in cache. It's also why shaving a few bytes off a data structure could pay huge dividends even though the memory footprint barely changes. If it fits neatly in the cache, it will be faster.

The only way to know if you have a performance problem related to caching is to benchmark very carefully. Remember each processor type has varying amounts of cache, so what might work well on your i7 CPU might be relatively terrible on an i5.

It's only in extremely performance sensitive applications that the cache really becomes something you worry about. For example, if you need to maintain a steady 60FPS frame rate in a game, you'll be looking at cache problems constantly. Every millisecond counts here. Likewise, anything that runs the CPU at 100% for extended periods of time, such as rendering video, will want to pay very close attention to how much they could gain from adjusting the code that's emitted.

You do have control over how your code is generated with compiler flags. Some will produce smaller code, some theoretically faster by unrolling loops and other tricks. To find the optimal setting can be a very time-consuming process. Likewise, you'll need to pay very careful attention to your data structures and how they're used.