Define a new type of memory

Question

Is it possible to define a new type of memory. For example in some embedded systems compilers you can do this:

__flash const char array[] = "This is memory in flash (not ram) so I can waste it!"; 

So is it possible to go even crazier and define a new type of memory (like say an SD card).

I am basically asking is it possible to define what an SD card is (how to access data in it) and then declare a variable in the sd memory. (every where it sees a write it calls the sd method, every where it sees a read it calls the sd method):

class SDCard{ public:   void write(void* data,size_t length);   void* read(size_t length);   void* memAlloc(size_t length);  };  __sd char myVar;  //Grabs an address in the sd card based on SDCard::memAlloc myVar = 'A';  //Calls whatever I defined the sd card write operation as char other = myVar;  //Calls whatever I defined the sd card read operation as 

I am using gcc if I can do something special with that (I am almost willing to modify the compiler source to allow me to do this).

Something like this is possible, but there are a few issues:

struct Vol_t{   //Would represent an SD card an external RAM or any other memory     void write(void* data,size_t len,size_t add) const{}     void* read(size_t len,size_t add) const{} }; template<Vol_t* sd, class Type,size_t address> struct MemDef{  //Wrap your type with this (maybe add -> operator support later     void operator=(Type&& data){         sd->write(&data,sizeof(data),address);     }     operator Type&(){         return *reinterpret_cast<Type*>(sd->read(sizeof(Type),address));     } };  Vol_t SD;  //Initialize our SD card MemDef<&SD,int,0xdeadbeaf> sdVar;  //Declare an int variable on the SD card  int main(int argc,char** args){     sdVar = 12;   //Writes to the SD card     int local = sdVar;  //Reads from the SD card     system("Pause"); } 

Problems with this approach:

• Optimizer must do every single read/write----it cannot do any optimizations on a variable declared in this way. (this is the major issue)
• It is a little inelegant (but it does get the job done)
• You have to specify what memory address to use (It would be awesome if the compiler could somehow figure all of that out before compilation and automatically

So maybe I have to add a keyword to gcc (if that is the case please point me in the right direction to get started).

Edit: there is another problem with this approach. If a type has a pointer to another variable that variable will not be initialized on the SD card (only the pointer will).

Answer 1

There could be two cases:

1. The hardware is such that a certain range of addresses maps to the SD card, and data can be written/read from the SD card using normal memory access instructions, within that range. Regular pointers are used.
2. Special instructions/functions need to be used to read/write to the SD card. For example an SDCard::read function, which calls a special function on the OS (if any), or an interrupt instruction.

__flash is a GCC extension. It uses a different instruction to access the memory, and locates static data on another segment. But it cannot be generalized in this way using C++ alone. It also can't be used with dynamic allocation.

First case (range of addresses)

To use regular pointers to read/write data to the SD card, they need to be marked volatile. This way the compiler does not optimize away and reads/writes. volatile means the memory may be changed/be used from outside the program, like for instance the hardware writing it to a SD card. See http://en.cppreference.com/w/cpp/language/cv .

For example

volatile char* data = 0x00010000; memcpy(data, "test", 5); 

writes "test" on the SD card, if for example the memory range 0x00010000..0x0001ffff maps to it.

To dynamically allocate memory on the SD card, like with malloc and free for regular working memory, a custom allocator would be needed. It would need to handle memory segmentation itself i.e. it needs to map which areas of the memory are free or allocated, and allocate(len) needs to find a free segment of length at least len. This is normally handled by the operating system.

This can be written in C++ as an allocator class, a class which must fulfill the requirements of Allocator (concept): http://en.cppreference.com/w/cpp/concept/Allocator . For example (incomplete):

template<typename T> class SD_allocator {     using value_type = T;     using pointer = T*;     pointer allocate(std::size_t len) {} // allocate segment of len bytes     void deallocate(pointer, std::size_t) {} }; 

If can then be used with STL containers, like:

std::vector<int, SD_allocator<int>> vec; 

uses the memory on the SD card for the items of vec. Here they are non-volatile, and for meant for use inside the program only, not for persistent storage on the SD card.

The standard allocator in C++ is std::allocator, which allocates regular memory like malloc and free.

Boost seems to provide an allocator which handles segmentation, on a custom defined memory region:

http://www.boost.org/doc/libs/1_35_0/doc/html/interprocess/managed_memory_segments.html http://www.boost.org/doc/libs/1_55_0/doc/html/boost/interprocess/allocator.html

For persistent storage, such as an SD card, it may be better to define a fixed structure and layout for the data on the SD card, and then read/write to it.

struct SDCard_data {     std::int32_t int1;     std::int32_t buffer1[500];     std::int8_t padding_[34];     int four_bit1 : 4;     int four_bit2 : 4;     bool bit1:1;     bool bit2:1;     bool bit3:1;     bool bit4:1; };  static volatile SDCard_data* sdcard     = reinterpret_cast<volatile SDCard_data*>(0x0001000);  int write_to_card() {     // writes to the card     card->int1 = 32;     card->bit3 = true; } 

Second case (special instructions)

If reads/writes to the SD card do not correspond to regular memory access instructions on the hardware, data on it cannot be accessed directly using raw volatile pointers.

If the goal is to still access it in that way, a class such as MemDef would be needed. It may be better to treat the SD card like a file/stream, and instead write/read whole chunks of data to/from it, using functions like fopen, fread, fprintf or similar.

Objects need to be serialized/unserialized for this. Copying a struct as raw memory, like

struct A; A a; write_to_card(reinterpret_cast<void*>(&a), sizeof(A)) 

works as long as the struct is a PODType and does not contain any pointers/references, i.e. types whose internal representation depends on the memory addresses. It is also dependent on the platform's memory layout (alignment, struct padding), endianness, representation of float, CHAR_BIT, etc. For cross-platform support (e.g. when the SD-card is read from another device with another microcontroller, some file format would need to be used instead.

It may also be possible (but difficult) to define a custom Allocator class, which uses a class like MemDef as pointer type.