Sharing data system wide

Question

Good evening. I'm looking for a method to share data from my application system-wide, so that other applications could read that data and then do whatever they want with it (e.g. format it for display, use it for logging, etc). The data needs to be updated dynamically in the method itself.

WMI came to mind first, but then you've got the issue of applications pausing while reading from WMI. Additionally, i've no real idea how to setup my own namespace or classes if that's even possible in Delphi.

Using files is another idea, but that could get disk heavy, and it's a real awful method to use for realtime data.

Using a driver would probably be the best option, but that's a little too intrusive on the users end for my liking, and i've no idea on where to even start with it.

WM_COPYDATA would be great, but i'm not sure if that's dynamic enough, and whether it'll be heavy on resources or not.

Using TCP/IP would be the best choice for over the network, but obviously is of little use when run on a single system with no networking requirement.

As you can see, i'm struggling to figure out where to go with this. I don't want to go into one method only to find that it's not gonna work out in the end. Essentially, something like a service, or background process, to record data and then allow other applications to read that data. I'm just unsure on methods. I'd prefer to NOT need elevation/UAC to do this, but if needs be, i'll settle for it.

I'm running in Delphi 2010 for this exercise.

Any ideas?

Answer 1

You want to create some Client-Server architecture, which is also called IPC.

Using WM_COPYDATA is a very good idea. I found out it is very fast, lightweight, and efficient on a local machine. And it can be broadcasted over the system, to all applications at once (to be used with care if some application does not handle it correctly).

You can also share some memory, using memory mapped files. This is may be the fastest IPC option around for huge amount of data, but synchronization is a bit complex (if you want to share more than one buffer at once).

Named pipes are a good candidates for local. They tend to be difficult to implement/configure over a network, due to security issues on modern Windows versions (and are using TCP/IP for network communication - so you should better use directly TCP/IP instead).

My personal advice is that you shall implement your data sharing with abstract classes, able to provide several implementations. You may use WM_COPYDATA first, then switch to named pipes, TCP/IP or HTTP in order to spread your application over a network.

For our Open Source Client-Server ORM, we implemented several protocols, including WM_COPY_DATA, named pipe, HTTP, or direct in-process access. You can take a look at the source code provided for implementation patterns. Here are some benchmarks, to give you data from real implementations:

 Client server access: 
  - Http client keep alive: 3001 assertions passed
     first in 7.87ms, done in 153.37ms i.e. 6520/s, average 153us
  - Http client multi connect: 3001 assertions passed
     first in 151us, done in 305.98ms i.e. 3268/s, average 305us
  - Named pipe access: 3003 assertions passed
     first in 78.67ms, done in 187.15ms i.e. 5343/s, average 187us
  - Local window messages: 3002 assertions passed
     first in 148us, done in 112.90ms i.e. 8857/s, average 112us
  - Direct in process access: 3001 assertions passed
     first in 44us, done in 41.69ms i.e. 23981/s, average 41us
  Total failed: 0 / 15014  - Client server access PASSED

As you can see, fastest is direct access, then WM_COPY_DATA, then named pipes, then HTTP (i.e. TCP/IP). Message was around 5 KB of JSON data containing 113 rows, retrieved from server, then parsed on the client 100 times (yes, our framework is fast :) ). For huge blocks of data (like 4 MB), WM_COPY_DATA is slower than named pipes or HTTP-TCP/IP.

Answer 2

Where are several IPC (inter-process communication) methods in Windows. Your question is rather general, I can suggest memory-mapped files to store your shared data and message broadcasting via PostMessage to inform other application that the shared data changed.