Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

USB Mass Storrage on File Level

The Challenge: I have an Linux hand-held device, which records data and stores it to the disc. It should exchange these data with a Windows application via USB. When this data is accessible by the user - e.g. via USB-mass-storage - it has to be encrypted. It should work out-of-the-box, with a variety of OS, also for citrix terminal sessions, etc.

The Plan: I create a file-system in user-space with FUSE, and offer it to windows via mass-storage. Whenever Windows accesses one file, I get a callback and encrypt the data on the fly. Further more, we can have some dynamic content - e.g. when some password is written into a file, more content is shown.

The Problem: When using the mass-storage gadget (e.g. g_file_storage) it only accepts files or block-devices - but no file-systems (directories). Why?

[...] it provides a simple interface to read and write sectors of data—much like the low-level interface used to access any hard drive [...]. Operating systems may treat the USB drive like a hard drive, and can format it with any file system they like. (from wikipedia)

So there is no chance to have a dynamic file-system via mass-storage... and this seems to be the reason, why my android mobile phone un-mounts all data on the phone, when I connect it to the PC.

Options:

  • Create a 'block-device in userspace' - similar to FUSE (need a reverse-FAT-driver, when I want to offer files dynamically)
  • Implement my own nbd-server to create a block-device (need a reverse-FAT-driver as well?)
  • I save encrypted files to a partition, and pass this partition to the mass-storage gadget (Problem would be the performance and the lack of dynamic interaction)
  • Do not offer a mass-storrage device and watch out for other ideas (eth over USB)

At the moment, only the last option seems to be realistic - or do you have another tip for me?

I would be grateful!

Charly

like image 931
Charly Avatar asked May 16 '11 20:05

Charly


People also ask

What is a USB mass storage device in Windows 10?

A USB mass storage device is plug & play but a driver is still involved. It’s just installed automatically by Windows 10 and users don’t notice it. In some cases though, the driver fails to install, or the installation is interrupted. Windows 10 generally doesn’t attempt to install the driver again. If you see the USB Mass Storage Device has ...

How do I fix a USB mass storage device that hasn't installed?

A USB mass storage device that hasn’t installed properly may need to be uninstalled and then reinstalled to work. Open Device Manager. Expand Other Devices. Right-click the USB mass storage device that is listed. Select Uninstall device from the context menu. Once uninstalled, connect the USB mass storage device to your system.

What is the USB mass storage protocol?

The USB mass storage protocol is a block device protocol; it does not operate at the level of files or directories. The Windows host expects to see a raw VFAT filesystem exposed by the g_mass_storage driver, and will make writes and reads to VFAT metadata as appropriate to figure out how directories are structured.

What are the common USB mass storage device error messages?

Perhaps, you may find an error message like Unknown USB Device (Device Descriptor Request Failed), Unknown USB Device (Port Reset Failed), Unknown USB Device (Set Address Failed), etc. At times, you may see that there is a yellow mark next to the USB mass storage device.


2 Answers

The USB mass storage protocol is a block device protocol; it does not operate at the level of files or directories. The Windows host expects to see a raw VFAT filesystem exposed by the g_mass_storage driver, and will make writes and reads to VFAT metadata as appropriate to figure out how directories are structured.

Because of this, it's nontrivial to expose a FUSE filesystem to the windows host. You'd have to emulate VFAT, assigning blocks in the virtual filesystem to metadata and to data, and since the windows host is free to cache any data or metadata it reads, once you assign some metadata or data it cannot change (so changes to your FUSE data could not be reflected in the windows filesystem). The windows host can also delay and reorder writes to both metadata and data - it's all real mess if you try to emulate.

Now, there are some things you can do:

  1. You can write a custom IFS driver on the windows side to interact with your linux device over a custom protocol that works at the file/directory level.
  2. You can treat the USB device as a virtual ethernet port, and speak CIFS to the windows host
  3. You can somehow create a static VFAT layout on connection time to expose to the windows host; not-yet-decryptable data can return I/O errors in order to avoid the Windows host caching raw encrypted data.
  4. You can just encrypt a raw block device using dm-crypt, and expose this entire block device (encrypted as one chunk) to windows.
  5. You could implement a MTP gadget.

These approaches come with their own problems:

  1. Requires a windows driver to be installed, and to be signed by microsoft etc. Can't be used on a machine without administrative access to install the driver.
  2. Won't autoplay; the user would need to browse through the network browser to get access to the files. Firewall settings may interfere. May have significant overhead.
  3. Very complex. Handling metadata updates on the backend may be extremely difficult. Surprise unplug events may be devastating. Windows behavior on receiving an IO error may be a problem if the user tries to access a locked file.
  4. No file-level encryption is available, but otherwise should work well.
  5. I'm unsure exactly how much support for non-media files MTP has; support is not as widespread as mass storage support.
like image 150
bdonlan Avatar answered Oct 20 '22 11:10

bdonlan


It may interest you to know that Android, which previously exposed itself as a USB mass storage device to the host, is instead acting as an MTP device (as of Honeycomb).

That being said, somebody has already implemented your option 1, though with the "device" and "host" a bit reversed. QEMU has an insane hack called vvfat which is able to create a fake block device which to the VM looks like it contains a VFAT filesytem just from a directory/file tree on the host. It requires a full recursive scan before starting, is dependent on details of how OSes write to filesystems, and falls over if you independently change any files while it's in use, but (somehow) manages to (sometimes) work.

like image 34
ephemient Avatar answered Oct 20 '22 12:10

ephemient