Python Pandas Find All Rows Where All Values Are Nan

So I have a dataframe with 5 columns. I would like to pull the indices where all of the columns are NaN. I was using this code: nan = pd.isnull(df.all) but that is just returning

Solution 1:

It should just be:

df.isnull().all(1)

The index can be accessed like:

df.index[df.isnull().all(1)]

Demonstration

np.random.seed([3,1415])
df = pd.DataFrame(np.random.choice((1, np.nan), (10, 2)))
df

idx = df.index[df.isnull().all(1)]
nans = df.ix[idx]
nans

---

Timing

code

np.random.seed([3,1415])
df = pd.DataFrame(np.random.choice((1, np.nan), (10000, 5)))

Solution 2:

Assuming your dataframe is named df, you can use boolean indexing to check if all columns (axis=1) are null. Then take the index of the result.

np.random.seed(0)
df = pd.DataFrame(np.random.randn(5,3))
df.iloc[-2:,:]= np.nan
>>> df
          01201.7640520.4001570.97873812.2408931.867558-0.97727820.950088-0.151357-0.1032193NaNNaNNaN4NaNNaNNaN

nan = df[df.isnull().all(axis=1)].index

>>> nan
Int64Index([3,4], dtype='int64')

Solution 3:

From the master himself: https://stackoverflow.com/a/14033137/6664393

nans = pd.isnull(df).all(1).nonzero()[0]