numpy.memmap for an array of strings?

Question

Is it possible to use numpy.memmap to map a large disk-based array of strings into memory?

I know it can be done for floats and suchlike, but this question is specifically about strings.

I am interested in solutions for both fixed-length and variable-length strings.

The solution is free to dictate any reasonable file format.

Answer 1

If all the strings have the same length, as suggested by the term "array", this is easily possible:

a = numpy.memmap("data", dtype="S10")

would be an example for strings of length 10.

Edit: Since apparently the strings don't have the same length, you need to index the file to allow for O(1) item access. This requires reading the whole file once and storing the start indices of all strings in memory. Unfortunately, I don't think there is a pure NumPy way of indexing without creating an array the same size as the file in memory first. This array can be dropped after extracting the indices, though.