{"id":530,"date":"2024-04-03T17:40:50","date_gmt":"2024-04-03T15:40:50","guid":{"rendered":"https:\/\/bitwise.exposed\/?p=530"},"modified":"2026-03-16T09:38:47","modified_gmt":"2026-03-16T08:38:47","slug":"fill-missing-data-with-mode","status":"publish","type":"post","link":"https:\/\/bitwise.exposed\/index.php\/2024\/04\/03\/fill-missing-data-with-mode\/","title":{"rendered":"Fill missing data, categorical & numerical"},"content":{"rendered":"\n

Cookbook | Data Cleaning<\/em><\/mark><\/h4>\n\n\n\n
\"\"<\/figure>
\n

Published and available on GitHub as part of the Python Cookbook Repository: fillna_cat_num_values.ipynb<\/a><\/p>\n<\/div><\/div>\n\n\n\n

\n\n\n\n

Filling missing categorical and numerical data<\/mark> with fillna(), mode()<\/mark> and statistical measures like mean() or median()<\/mark>.<\/p>\n\n\n\n

\n\n\n\n
Topics<\/h5>\n\n\n\n
UseCase<\/strong><\/td>Cookbook recipe<\/strong><\/td><\/tr>
Categorical Data<\/a><\/td>Select Columns in DataFrame manually. Fill NA\/NaN with ‚mode‘, in case of equal take first index.<\/td><\/tr>
Numerical Data<\/a><\/td>Select Columns in DataFrame manually. Fill NA\/Nan with mean.’Select Columns in DataFrame manually. Fill NA\/Nan with ‚mean‘.<\/td><\/tr>
Mixed Data<\/a><\/td>Select Columns in DataFrame by Dtype. Fill NA\/NaN with ‚mode or mean‘.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
\n\n\n\n
UseCase: categorical data<\/h5>\n\n\n\n

Demo data <\/strong>
is a pandas dataframe with categorical data and missing values.

Objective<\/strong>
is to fill the missing values, with the most occurring value.<\/p>\n\n\n\n

Notes<\/strong>
In case we have equally occurring values, we just pick the first.
–> .mode()[0]<\/p>\n\n\n\n

Python<\/span><\/path><\/path><\/svg><\/span>
import<\/span> pandas <\/span>as<\/span> pd<\/span><\/span>\nimport<\/span> numpy <\/span>as<\/span> np<\/span><\/span>\n<\/span>\n# Create simple dictionary with some missing values<\/span><\/span>\ndata = {<\/span><\/span>\n    <\/span>'Name'<\/span>: [<\/span>'John'<\/span>, <\/span>'Anna'<\/span>, np.nan, <\/span>'Linda'<\/span>, <\/span>'John'<\/span>],<\/span><\/span>\n    <\/span>'Type'<\/span>: [<\/span>'Type1'<\/span>, <\/span>'Type2'<\/span>, <\/span>'Type2'<\/span>, np.nan, <\/span>'Type1'<\/span>],<\/span><\/span>\n    <\/span>'Country'<\/span>: [<\/span>'Country1'<\/span>, np.nan, <\/span>'Country2'<\/span>, <\/span>'Country1'<\/span>, <\/span>'Country2'<\/span>],<\/span><\/span>\n}<\/span><\/span>\n<\/span>\n# Convert dictionary to pandas DataFrame<\/span><\/span>\ndata = pd.DataFrame(data)<\/span><\/span>\n<\/span>\nprint<\/span>(<\/span>"Original DataFrame:"<\/span>)<\/span><\/span>\nprint<\/span>(data)<\/span><\/span>\n<\/span>\n# Apply mode and fillna<\/span><\/span>\nfor<\/span> column <\/span>in<\/span> [<\/span>'Name'<\/span>, <\/span>'Type'<\/span>, <\/span>'Country'<\/span>]:<\/span><\/span>\n    mode = data[column].mode()[<\/span>0<\/span>] <\/span># in case we have more then one mode<\/span><\/span>\n    data[column].fillna(mode, <\/span>inplace<\/span>=<\/span>True<\/span>)<\/span><\/span>\n<\/span>\nprint<\/span>(<\/span>"<\/span>\\n<\/span>DataFrame after filling NA values with mode:"<\/span>)<\/span><\/span>\nprint<\/span>(data)<\/span><\/span>\n<\/span><\/code><\/pre><\/div>\n\n\n\n
Output:<\/p>\n\n\n\n
Python<\/span><\/path><\/path><\/svg><\/span>
# Original DataFrame:<\/span><\/span>\n<\/span>\n    Name   Type   Country<\/span><\/span>\n0<\/span>   John  Type1  Country1<\/span><\/span>\n1<\/span>   Anna  Type2       NaN<\/span><\/span>\n2<\/span>    NaN  Type2  Country2<\/span><\/span>\n3<\/span>  Linda    NaN  Country1<\/span><\/span>\n4<\/span>   John  Type1  Country2<\/span><\/span>\n<\/span>\n<\/span>\n<\/span>\n# DataFrame after filling NA values with mode:<\/span><\/span>\n<\/span>\n    Name   Type   Country<\/span><\/span>\n0<\/span>   John  Type1  Country1<\/span><\/span>\n1<\/span>   Anna  Type2  Country1<\/span><\/span>\n2<\/span>   John  Type2  Country2<\/span><\/span>\n3<\/span>  Linda  Type1  Country1<\/span><\/span>\n4<\/span>   John  Type1  Country2<\/span><\/span>\n<\/span><\/code><\/pre><\/div>\n\n\n\n
\n\n\n\n
UseCase: numerical data<\/h5>\n\n\n\n
Demo data <\/strong>
is a pandas dataframe with numerical data and missing values.

Objective<\/strong>
is to fill the missing values, with the median<\/p>\n\n\n\n
Python<\/span><\/path><\/path><\/svg><\/span>
import<\/span> pandas <\/span>as<\/span> pd<\/span><\/span>\nimport<\/span> numpy <\/span>as<\/span> np<\/span><\/span>\n<\/span>\n# Sample data with some missing values<\/span><\/span>\ndata = {<\/span><\/span>\n    <\/span>'Turnover'<\/span>: [<\/span>100<\/span>, <\/span>200<\/span>, np.nan, <\/span>400<\/span>, <\/span>500<\/span>, <\/span>600<\/span>, np.nan, <\/span>800<\/span>],<\/span><\/span>\n    <\/span>'Transactions'<\/span>: [<\/span>1<\/span>, <\/span>2<\/span>, <\/span>3<\/span>, np.nan, np.nan, <\/span>6<\/span>, <\/span>7<\/span>, <\/span>8<\/span>]<\/span><\/span>\n}<\/span><\/span>\n<\/span>\n# Convert dictionary to a pandas DataFrame<\/span><\/span>\ndata = pd.DataFrame(data)<\/span><\/span>\n<\/span>\nprint<\/span>(<\/span>"Original DataFrame:"<\/span>)<\/span><\/span>\nprint<\/span>(data)<\/span><\/span>\n<\/span>\n# Fill missing values with column median<\/span><\/span>\nfor<\/span> column <\/span>in<\/span> [<\/span>'Turnover'<\/span>, <\/span>'Transactions'<\/span>]:<\/span><\/span>\n    median = data[column].median()<\/span><\/span>\n    data[column].fillna(median, <\/span>inplace<\/span>=<\/span>True<\/span>)<\/span><\/span>\n<\/span>\nprint<\/span>(<\/span>"<\/span>\\n<\/span>DataFrame after filling NA values with median:"<\/span>)<\/span><\/span>\nprint<\/span>(data)<\/span><\/span>\n<\/span><\/code><\/pre><\/div>\n\n\n\n
Output:<\/p>\n\n\n\n
Python<\/span><\/path><\/path><\/svg><\/span>
# Original DataFrame:<\/span><\/span>\n<\/span>\n   Turnover  Transactions<\/span><\/span>\n0<\/span>     <\/span>100.0<\/span>           <\/span>1.0<\/span><\/span>\n1<\/span>     <\/span>200.0<\/span>           <\/span>2.0<\/span><\/span>\n2<\/span>       NaN           <\/span>3.0<\/span><\/span>\n3<\/span>     <\/span>400.0<\/span>           NaN<\/span><\/span>\n4<\/span>     <\/span>500.0<\/span>           NaN<\/span><\/span>\n5<\/span>     <\/span>600.0<\/span>           <\/span>6.0<\/span><\/span>\n6<\/span>       NaN           <\/span>7.0<\/span><\/span>\n7<\/span>     <\/span>800.0<\/span>           <\/span>8.0<\/span><\/span>\n<\/span>\n<\/span>\n<\/span>\n# DataFrame after filling NA values with median:<\/span><\/span>\n<\/span>\n   Turnover  Transactions<\/span><\/span>\n0<\/span>     <\/span>100.0<\/span>           <\/span>1.0<\/span><\/span>\n1<\/span>     <\/span>200.0<\/span>           <\/span>2.0<\/span><\/span>\n2<\/span>     <\/span>450.0<\/span>           <\/span>3.0<\/span><\/span>\n3<\/span>     <\/span>400.0<\/span>           <\/span>4.5<\/span><\/span>\n4<\/span>     <\/span>500.0<\/span>           <\/span>4.5<\/span><\/span>\n5<\/span>     <\/span>600.0<\/span>           <\/span>6.0<\/span><\/span>\n6<\/span>     <\/span>450.0<\/span>           <\/span>7.0<\/span><\/span>\n7<\/span>     <\/span>800.0<\/span>           <\/span>8.0<\/span><\/span>\n<\/span><\/code><\/pre><\/div>\n\n\n\n
UseCase: Mixed data (numerical, categorical)<\/p>\n\n\n\n
Python<\/span><\/path><\/path><\/svg><\/span>
import<\/span> pandas <\/span>as<\/span> pd<\/span><\/span>\nimport<\/span> numpy <\/span>as<\/span> np<\/span><\/span>\n<\/span>\n# Simple DataFrame with both numerical and categorical data<\/span><\/span>\ndata = {<\/span><\/span>\n    <\/span>'Age'<\/span>: [<\/span>25<\/span>, <\/span>30<\/span>, <\/span>35<\/span>, np.nan, <\/span>45<\/span>],<\/span><\/span>\n    <\/span>'City'<\/span>: [<\/span>'New York'<\/span>, <\/span>'Seattle'<\/span>, <\/span>'San Francisco'<\/span>, <\/span>'Austin'<\/span>, np.nan],<\/span><\/span>\n    <\/span>'Income'<\/span>: [<\/span>50000<\/span>, <\/span>70000<\/span>, np.nan, <\/span>90000<\/span>, <\/span>100000<\/span>]<\/span><\/span>\n}<\/span><\/span>\n<\/span>\ndf = pd.DataFrame(data)<\/span><\/span>\nprint<\/span>(<\/span>"Original DataFrame:"<\/span>)<\/span><\/span>\nprint<\/span>(df)<\/span><\/span>\n<\/span>\ndef<\/span> <\/span>fill_na_with_mean<\/span>(<\/span>df<\/span>, <\/span>num_cols<\/span>):<\/span><\/span>\n    mean_values = df[num_cols].mean()<\/span><\/span>\n    df[num_cols] = df[num_cols].fillna(mean_values)<\/span><\/span>\n    <\/span>return<\/span> df<\/span><\/span>\n<\/span>\n<\/span>\ndef<\/span> <\/span>fill_na_with_mode<\/span>(<\/span>df<\/span>, <\/span>cat_cols<\/span>):<\/span><\/span>\n    mode_values = df[cat_cols].mode().iloc[<\/span>0<\/span>]<\/span><\/span>\n    df[cat_cols] = df[cat_cols].fillna(mode_values)<\/span><\/span>\n    <\/span>return<\/span> df<\/span><\/span>\n<\/span>\n<\/span>\ndef<\/span> <\/span>clean_data<\/span>(<\/span>df<\/span>):<\/span><\/span>\n    <\/span># Identify numerical columns and fill NA\/NaN with mean<\/span><\/span>\n    num_cols = df.select_dtypes(<\/span>include<\/span>=np.number).columns<\/span><\/span>\n    df = fill_na_with_mean(df, num_cols)<\/span><\/span>\n<\/span>\n    <\/span># Identify categorical columns and fill NA\/NaN with mode<\/span><\/span>\n    cat_cols = df.select_dtypes(<\/span>include<\/span>=<\/span>'object'<\/span>).columns<\/span><\/span>\n    df = fill_na_with_mode(df, cat_cols)<\/span><\/span>\n<\/span>\n    <\/span>return<\/span> df<\/span><\/span>\n<\/span>\n<\/span>\n<\/span>\n# Apply the function on our DataFrame<\/span><\/span>\ncleaned_df = clean_data(df)<\/span><\/span>\nprint<\/span>(<\/span>"<\/span>\\n<\/span>Cleaned DataFrame:"<\/span>)<\/span><\/span>\nprint<\/span>(cleaned_df)<\/span><\/span>\n<\/span><\/code><\/pre><\/div>\n\n\n\n
Output:<\/p>\n\n\n\n
Python<\/span><\/path><\/path><\/svg><\/span>
# Original DataFrame:<\/span><\/span>\n<\/span>\n    Age           City    Income<\/span><\/span>\n0<\/span>  <\/span>25.0<\/span>       New York   <\/span>50000.0<\/span><\/span>\n1<\/span>  <\/span>30.0<\/span>        Seattle   <\/span>70000.0<\/span><\/span>\n2<\/span>  <\/span>35.0<\/span>  San Francisco       NaN<\/span><\/span>\n3<\/span>   NaN         Austin   <\/span>90000.0<\/span><\/span>\n4<\/span>  <\/span>45.0<\/span>            NaN  <\/span>100000.0<\/span><\/span>\n<\/span>\n<\/span>\n# Cleaned DataFrame:<\/span><\/span>\n<\/span>\n     Age           City    Income<\/span><\/span>\n0<\/span>  <\/span>25.00<\/span>       New York   <\/span>50000.0<\/span><\/span>\n1<\/span>  <\/span>30.00<\/span>        Seattle   <\/span>70000.0<\/span><\/span>\n2<\/span>  <\/span>35.00<\/span>  San Francisco   <\/span>77500.0<\/span><\/span>\n3<\/span>  <\/span>33.75<\/span>         Austin   <\/span>90000.0<\/span><\/span>\n4<\/span>  <\/span>45.00<\/span>         Austin  <\/span>100000.0<\/span><\/span>\n<\/span><\/code><\/pre><\/div>\n\n\n\n
mode()<\/h5>\n\n\n\n
The mode()<\/code> function<\/mark> is part of the pandas library in Python. It is used to find <\/mark>the mode (most frequently occurring value<\/mark>) in a pandas <\/mark>Series <\/mark>or DataFrame<\/mark>.<\/p>\n\n\n\n
fillna()<\/h5>\n\n\n\n
The fillna() function<\/mark> is part of the pandas library in Python. This function is used to fill NA\/NaN values<\/mark> using the specified method.<\/p>\n","protected":false},"excerpt":{"rendered":"
Filling missing categorical and numerical data with fillna(), mode() and statistical measures like mean() or median().<\/p>\n","protected":false},"author":1,"featured_media":581,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[29,28,26,13,27],"tags":[],"class_list":["post-530","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-cleaning","category-data-engineering","category-pandas","category-python","category-statistics"],"_links":{"self":[{"href":"https:\/\/bitwise.exposed\/index.php\/wp-json\/wp\/v2\/posts\/530","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/bitwise.exposed\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/bitwise.exposed\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/bitwise.exposed\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/bitwise.exposed\/index.php\/wp-json\/wp\/v2\/comments?post=530"}],"version-history":[{"count":30,"href":"https:\/\/bitwise.exposed\/index.php\/wp-json\/wp\/v2\/posts\/530\/revisions"}],"predecessor-version":[{"id":621,"href":"https:\/\/bitwise.exposed\/index.php\/wp-json\/wp\/v2\/posts\/530\/revisions\/621"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/bitwise.exposed\/index.php\/wp-json\/wp\/v2\/media\/581"}],"wp:attachment":[{"href":"https:\/\/bitwise.exposed\/index.php\/wp-json\/wp\/v2\/media?parent=530"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/bitwise.exposed\/index.php\/wp-json\/wp\/v2\/categories?post=530"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/bitwise.exposed\/index.php\/wp-json\/wp\/v2\/tags?post=530"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}