data & it depends
{ bitwise.exposed } is a blog about data engineering and data science
with focus on business intelligence and BI topics like
data warehouse, architecture, models, querying, performance metrics …
-
Dealing with outliers – IQR method
Cookbook | Data Cleaning Published and available on GitHub as part of the Python Cookbook Repository: outliners_IQR_method.ipynb Dealing with outliers in numerical columns – using IQR method The Interquartile Range (IQR) method is a statistical technique used to identify outliers in a dataset. The IQR…
-
Fill missing data, categorical & numerical
Filling missing categorical and numerical data with fillna(), mode() and statistical measures like mean() or median().
-
If metrics are unclear to stakeholders
Identify issues Identifying the underlying reason behind unclear success metrics is the initial step towards resolving the issue. First, determine if the lack of clarity and consensus on success metrics stems from misunderstandings or missing definitions, as this would indicate a political issue. If so,…
-
Populate a fact table with surrogate keys from dimensions
Populating a fact table with surrogate keys from dimension tables involves connecting the dimension tables to the fact table through foreign key relationships and inserting the corresponding surrogate keys into the fact table. Let’s start with some constraints: Reinforce the relationships between the tables: Load…
-
Dimensional Modeling – Pros & Cons
The key idea behind dimensional modeling is to create a data structure that is easy to understand and navigate, while providing high performance for analytical queries. The star schema, with a central fact table surrounded by dimension tables, even offers a visual representation. This alignment…
-
Important factors in a Data Warehouse project.
You’re considering a data warehousing project. What are the most important factors to consider?
-
-
-
SCD2 simplified example
When dealing with Slowly Changing Dimension Type 2 (SCD2) in a data warehouse, a common practice is to use a staging pattern to handle the loading and processing of data before it is inserted into the final dimension table. This staging pattern helps manage historical…
-
Types of Dimension Tables
Dimension tables can be categorized into different types based on the nature of the data they represent. Here are some common types of dimension tables: Conformed Dimensions Conformed dimensions are dimensions that are shared and consistently used across multiple data marts or data warehouses within…
