Pandas: what is a NDFrame object (and what is a non-NDFrame object)

Question

I am trying to concat two DataFrames and am getting a 'TypeError: cannot concatenate a non-NDFrame object' error. I have been looking around, there are a lot of people getting this error, most of the time when they try to do something other than what their code really does, (so the answers solve their particular but unrelated questions)

My question is not to solve my particular problem, but to understand the error... What is a non-NDFrame object? Why can it not be concatenated?

Also, why is this about NDFrames (What are those, and where do I use them? Are all DataFrames NDFrames? Are there any NDFrames that are not DataFrames?)...

I would understand if it said 'TypeError: cannot concatenate a non-DataFrame object' (though I am not sure if them not being DataFrames was the cause of my error) This is mainly my curiosity questing, I am not trying to hunt any specific bug...

Answer 1

From the horse's mouth:

N-dimensional analogue of DataFrame. Store multi-dimensional in a size-mutable, labeled data structure

Then what's a DataFrame?

class DataFrame(NDFrame): Two-dimensional size-mutable, potentially heterogeneous tabular data structure with labeled axes (rows and columns). Arithmetic operations align on both row and column labels. Can be thought of as a dict-like container for Series objects.

As you can see, a DataFrame is a subclass (i.e. special case) of NDFrame. In Pandas programs generally, DataFrame is used a lot and NDFrame is used rarely. In fact, Pandas has Series for 1D, DataFrame for 2D, and for most people that's the end, even though half of Pandas' name is for Panel which Pandas also has, but most people do not use.

There is/was even a 4D thing in Pandas, but truly no one uses it (this being the internet, someone will now appear to say they do!). For higher dimensions than two or maybe three, some people have shifted their efforts to xarray. That's probably where it's at if your ambitions cannot be contained in 2D.