Pytorch RuntimeError: [enforce fail at CPUAllocator.cpp:56] posix_memalign(&data, gAlignment, nbytes) == 0. 12 vs 0

Question

I'm building a simple content based recommendations system. In order to compute the cosine similarity in a GPU accelerated way, i'm using Pytorch.

At the time of creating the tfidf vocabulary tensor from a csr_matrix, it promts the following RuntimeErrorr

RuntimeError: [enforce fail at CPUAllocator.cpp:56] posix_memalign(&data, gAlignment, nbytes) == 0. 12 vs 0

I'm doing it in this way:

coo = tfidf_matrix.tocoo()
values = coo.data
indices = np.vstack( (coo.row, coo.col ))
i = torch.LongTensor(indices)
v = torch.FloatTensor(values)
tfidf_matrix_tensor = torch.sparse.FloatTensor(i, v, torch.Size(coo1.shape)).to_dense() 
# Prompts the error

I tried with a small test (tfidf matrix size = 10,296) dataset and it works. The tfidf matrix size from the real dataset is (27639, 226957)

Answer 1

I tried the same piece of code that was throwing this error with the older version of PyTorch. It said that I need to have more RAM. Therefore, it's not a PyTorch bug. The only solution is to reduce matrix size somehow.