Why fetching data from Memory & I\O is expensive?

Question

When I read operating system based books everywhere its written that fetching data from memory and I\O (subsystem) is expensive considering time constraint and over-heads are high, and that's why in some hardware manufacturer provide some other way to access them as in ARM7 some ISAs like (load and store ) and device drivers made for RTOS are also of different kind considering this fact (uses RTDM) comparing GPOS, so if it's all true then I want to know the lower level details on this fact that why that memory and i\o transactions are expensive.

Answer 1

Retrieving data from memory and I/O devices is "costly" because of how many steps are involved, and each step adds a small amount of delay. Here is a generic example of the steps required to retrieve a value from memory:

1) Start with the data's virtual address in your program's memory space

2) Translate the virtual address into a memory physical address

3) Check the cache for the value

4) If the data isn't in the cache, make a request from the memory controller

5) Memory controller breaks the physical address into a row, column, bank and bits address

6) Memory controller reads data from DRAM - The act of actually retrieving data from DRAM is quite complex and has its own series of steps that can take time. See the Wikipedia entry for DRAM.

7) Return the data to the CPU

Keep in mind that every bit of communication that is done between different hardware units is done over buses, which always run at slower clock speeds than your CPU's nominal frequency.

Even this series of steps is greatly simplifying things. For example, Step 2 starts with looking in the translation-look-aside-buffer (TLB Wikipedia Entry), but if the the required translation isn't in the buffer anymore, then a second memory fetch must be performed just to get the physical address of the requested data.

Additionally, there's only so much RAM available in your system, so it's possible that the data you want has been removed from RAM and temporarily stored on your hard disk (called "swapping") if it hasn't been used in a while. Now what you thought was a simple RAM access is actually going to your hard disk first. By the way, this swapping can also happen to the tables that translate virtual addresses to physical addresses, so retrieving a single byte that hasn't been used in a while can actually turn into two disk accesses in the worst case.

Speaking of hard disks, I/O devices are a whole different story. The term itself covers a wide variety of devices: RS232 serial ports, USB, networks, external CD/DVD readers, external hard disks and solid-state drives, the list is endless. They each have different attributes and capabilities that makes them faster or slower than others, but the key is that none of them supply data as fast your CPU can consume it, due to their own attributes (such as waiting for hard disk platters to spin up), physical limitations on bus speeds and the number of translation layers data passes through. Therefore, every time your CPU makes a request from an I/O device, dozens, hundreds, or even thousands of clock cycles may pass before a reply is received.

The time your program can potentially have to wait for replies from these sources is why they are considered "expensive."