unit-3

data preparation steps

• loading

• cleaning

• transforming

• rearranging

analyst’s time for data preparation

• 80% or more

• 75%

• 40%

• 30%

NaN is also known as

• Sentinal value

• value

• both

• none

NAN full form

• Not a Number

• Not avaiable

• missing number

• none

s = pd.Series([2,3,np.nan,7,"The Hobbit"])

s.isnull()

• 0 False

  1 False

  2 True

  3 False

  4 False

  dtype: bool

• 0 False

  1 False

  2 False

  3 False

  4 False

  dtype: bool

• 0 False

  1 True

  2 False

  3 False

  4 False

  dtype: bool

• 0 False

  1 True

  2 True

  3 False

  4 False

  dtype: bool

data = pd.Series([1, NA, 3.5, NA, 7])

data.dropna()

• 0 1.0

  2 3.5

  3 7.0

• 0 1.0

  1 3.5

  2 7.0

• 0 1.0

  2 3.5

  4 7.0

• 0 1.0

  1 3.5

  4 7.0

data[data.notnull()] is equivalent to

• data.fillna()

• data.drop()

• data.dropna()

• none

data = pd.DataFrame([[1., 6.5, 3.], [1., NA, NA],[NA, NA, NA], [NA, 6.5, 3.]])

cleaned = data.dropna()

data

• cleaned DataFrame with missing values and prints it

• data DataFrame with incorrect values and prints it

• prints DataFrame with missing values

• prints data DataFrame without missing values

Passing how='all' in dropna()

• will only drop rows that are any NA

• will only drop rows that are all NA

• none

• both

d1 = { 'Name': ['Shark', 'Whale', 'Jellyfish', 'Starfish'], 'ID': [1, np.nan, 3, np.nan], 'Population': [100, 200, np.nan,200], }

df1 = pd.DataFrame(d1)

dfresult = df1.dropna()

print(dfresult)

• Name ID Pop

  0 Shark 1 100

• {'Name': ['Shark', 'Whale', 'Starfish'], 'ID': [1, 2, 4], 'Population': [100, 200, 200]}
• Name ID Pop

  0 Shark 1 100

  4 Starfish nan200