SECTION 1
Historical Context & Motivation
Data transformation has been a central concern in analytics engineering since the early days of relational databases, where schema enforcement at write time guaranteed type safety. As self-service BI tools proliferated in the 2010s, the responsibility for ensuring correct data types shifted from database administrators to analysts and data engineers working closer to the reporting layer. Microsoft's Power Query — originally released as an Excel add-in under the codename "Data Explorer" — was designed to bridge this gap by providing a functional, composable ETL engine that could infer and transform data types interactively. Understanding how Power Query handles type detection, explicit coercion, and the parsing failures that arise from locale mismatches or malformed data is essential for building reliable data pipelines in Power BI.
2013 • Power Query Preview for Excel
2015 • Power BI Desktop Launch
2018 • Dataflows in Power BI Service
2021 • Power Query Online Enhancements
The fundamental question this lesson addresses is deceptively simple: when Power Query reads a column of values from a CSV, database, or API, how does it decide what data type to assign, what happens when that assignment is wrong, and how should you intervene to ensure correctness? Answering this well requires understanding Power Query's type system, the M language's coercion semantics, and the locale-dependent parsing rules that silently corrupt dates and numbers when misconfigured.
SECTION 2
Core Principles & Definitions
Power Query's type system is richer than what many analysts expect from a GUI-based tool. Every column in a Power Query table has a type annotation — a metadata tag that tells the engine how to interpret the underlying values. When you load data from an untyped source like a CSV file, Power Query performs automatic type detection by sampling the first 200 rows and applying heuristics. This inference is convenient but frequently wrong, particularly for dates in ambiguous formats (is "01/02/2024" January 2 or February 1?), numeric strings with locale-specific thousand separators, or mixed-type columns. The core principles below form the foundation of reliable type management.
Primitive Types
type number, type text, type date, type datetime, type logical, type duration, and several others. Each maps to a specific CLR type in the underlying engine.Explicit vs. Implicit Coercion
Table.TransformColumnTypes or the UI's "Change Type" menu. Implicit coercion occurs when the "Changed Type" step is auto-generated on import. Always audit auto-generated steps.Locale-Sensitive Parsing
"en-US" culture strings.Error Propagation
Error value — not null. These errors propagate silently through subsequent steps and can corrupt aggregations if unhandled.Type Fidelity Across Sources
KEY TAKEAWAY
SECTION 3
Visual Explanation — The Type Coercion Pipeline
Error values that must be handled through replacement or conditional parsing strategies.The diagram above captures the fundamental tension in Power Query type management. On the happy path, values flow from raw text through detection and explicit casting to properly typed columns. However, any mismatch between the actual data format and the assumed parsing rules diverts values into the parsing failure branch. The critical insight is that Power Query's error values are not the same as nulls — they are sticky error objects that propagate through calculations and aggregations, meaning a single unhandled parsing failure in a date column can silently invalidate an entire time-series analysis. Disciplined type management requires moving from the auto-detect stage to the explicit cast stage as early as possible in your query, preferably with culture-specific overrides.
SECTION 4
How Type Coercion Works in M
Under the hood, every type change in the Power Query UI generates an M expression using the Table.TransformColumnTypes function. Understanding the function's signature and its overloads is essential for debugging locale-related parsing issues. The function accepts a table, a list of column-type pairs, and an optional culture parameter that controls how text is parsed into dates and numbers.
BASIC TYPE CHANGE (NO CULTURE)
This form uses the system locale of the machine or service running the query. On a US-locale system, "01/02/2024" parses as January 2; on a UK-locale system, it parses as February 1.
CULTURE-SPECIFIC TYPE CHANGE
The third argument "de-DE" forces German locale parsing: dots become thousand separators, commas become decimal separators, and dates follow DD.MM.YYYY. This is the deterministic approach — the query produces identical results regardless of where it runs.
ERROR-SAFE PARSING WITH TRY-OTHERWISE
The try...otherwise pattern wraps the parsing call so that unparseable values return null instead of an Error. This is analogous to exception handling in imperative languages. Use Date.FromText, Number.FromText, or Int64.From for granular, per-value coercion with explicit culture.
Critical: Error vs. Null Semantics
null. A null is a legitimate missing value that DAX functions like AVERAGE gracefully ignore. An Error is a fault marker that causes downstream DAX measures to return BLANK or fail entirely. Always convert errors to nulls explicitly using Table.ReplaceErrorValues or per-cell try...otherwise before loading data into the model.SECTION 5
Detailed Type Catalog & Common Parsing Pitfalls
Power Query supports a well-defined set of scalar types, each with its own parsing rules and failure modes. The table below catalogs the most frequently used types, their M-language identifiers, common source formats, and the pitfalls that arise during coercion. Recognizing these pitfalls before they produce errors is the hallmark of a well-constructed Power Query workflow.
| Type | M Identifier | Typical Raw Format | Common Pitfall |
|---|---|---|---|
| Whole Number | Int64.Type | "1234" | Thousand separators ("1,234") cause failure without correct locale |
| Decimal Number | type number | "3.14" | European sources use comma as decimal ("3,14"); US locale parses as 314 |
| Date | type date | "01/02/2024" | Ambiguous MM/DD vs DD/MM; silently swaps month/day without error |
| DateTime | type datetime | "2024-01-02T14:30:00Z" | Timezone offsets are dropped; use DateTimeZone type to preserve them |
| Text | type text | "Hello World" | Numbers stored as text prevent aggregation; leading zeros (ZIP codes) lost if cast to number |
| Logical | type logical | "TRUE" / "1" | Non-standard representations ("Yes", "Y", "1") require pre-cleaning before cast |
Power Query scalar types, identifiers, and common coercion pitfalls
The most insidious parsing bug is not an error but a silent misinterpretation. When the date "03/04/2024" is parsed under US locale, it becomes March 4; under UK locale, it becomes April 3. Both parsings succeed without any error flag. Unless you know the source system's date convention, you cannot detect this bug from the output alone. This is why explicitly specifying the culture parameter is not merely best practice — it is the only way to guarantee correctness when data crosses locale boundaries.
SECTION 6
Worked Example — Cleaning a Multi-Locale CSV
Consider a CSV file exported from a European ERP system. The file contains an order date column in DD/MM/YYYY format, a revenue column using commas as decimal separators and dots as thousand separators ("1.234,56"), a product name (text), and a status column containing "Yes"/"No" values that need to become logical. Some rows have "N/A" in the revenue column. The goal is to load this data into Power BI with correct types and no errors.
Cleaning European ERP Data in Power Query
- 1
Step 1 — Load and Inspect Raw Data
Connect to the CSV using "Get Data → Text/CSV". In the preview dialog, select a delimiter and load the data. Power Query auto-generates a "Changed Type" step based on the first 200 rows and the system locale. Open the Advanced Editor to inspect the generated M code. The auto-generated step likely misidentifies dates (parsing DD/MM as MM/DD) and fails on the European number format.Delete the auto-generated "Changed Type" step to start fresh. - 2
Step 2 — Replace Sentinel Values
Before casting types, replace non-parseable sentinel values. Select the Revenue column and use Transform → Replace Values to convert "N/A" tonull. In M, this generates:Table.ReplaceValue(Source, "N/A", null, Replacer.ReplaceValue, {"Revenue"}). This step is critical because "N/A" would produce an Error when cast to number, whereas null is handled gracefully by DAX aggregation functions.All "N/A" values replaced with null in Revenue column. - 3
Step 3 — Normalize Boolean Representations
The Status column contains "Yes"/"No" strings, which Power Query does not automatically cast to logical. Add a conditional column or useTable.TransformColumns(prev, {{"Status", each if _ = "Yes" then true else if _ = "No" then false else null}}). This explicit mapping avoids relying on the engine's limited boolean inference, which only recognizes "true"/"false" literals.Status column now contains true/false/null logical values. - 4
Step 4 — Apply Explicit Type Changes with Culture
Now apply the type change using the European locale. In the formula bar, enter:Table.TransformColumnTypes(prev, {{"OrderDate", type date}, {"Revenue", type number}, {"ProductName", type text}, {"Status", type logical}}, "fr-FR"). The "fr-FR" culture (which uses the same conventions as the ERP's German or French locale: DD/MM/YYYY dates, comma decimals) ensures that "03/04/2024" parses as April 3 and "1.234,56" parses as 1234.56. You can also use "de-DE" — the key is matching the source system's locale.All columns correctly typed: OrderDate = date, Revenue = decimal number, ProductName = text, Status = logical. - 5
Step 5 — Verify and Handle Residual Errors
Click the "Error" filter in the column header to inspect any remaining parsing failures. If errors exist, add:Table.ReplaceErrorValues(prev, {{"OrderDate", null}, {"Revenue", null}}). This converts any residual errors to null, ensuring clean loading. For production pipelines, consider logging error rows to a separate table for auditing rather than silently discarding them.Zero errors remaining. Data ready for Vertipaq compression and DAX analysis.
SECTION 7
Strengths, Limitations & Comparison of Type-Handling Approaches
Power Query offers several approaches to type management, each with distinct tradeoffs. Understanding these tradeoffs allows you to select the right strategy for a given scenario — for instance, using auto-detection for quick exploratory work but explicit culture-aware casting for production pipelines.
| Approach | Strengths | Limitations |
|---|---|---|
| Auto-Detection | Zero-effort; excellent for homogeneous, well-formatted sources; fast prototyping | Uses system locale (non-portable); samples only 200 rows; silently misparses ambiguous dates |
| Explicit Cast (no culture) | Deliberate type assignment per column; generated by UI "Change Type" menu | Still locale-dependent — behaves differently on US vs. European Power BI Service nodes |
| Explicit Cast (with culture) | Fully deterministic; portable across environments; correct handling of cross-locale data | Requires knowledge of source locale; all columns in one call share the same culture parameter |
| Per-Value Parsing Functions | Maximum granularity; per-column culture; supports try...otherwise for graceful failure | Verbose M code; slower for large datasets due to row-level evaluation; requires M proficiency |
| Source Schema Preservation | Database, OData, and Parquet sources carry type metadata; no parsing needed | Only works for typed sources; CSV, JSON, and Excel are untyped and require explicit handling |
Comparison of type-handling approaches in Power Query
KEY TAKEAWAY
SECTION 8
Connection to Advanced Theory — Custom Types & Structured Error Handling
The type-coercion fundamentals covered so far prepare you for more advanced Power Query patterns that become necessary as data pipelines scale. Two directions are particularly relevant: custom M types for schema enforcement across modular queries, and structured error handling patterns for building resilient data pipelines. Understanding these extensions elevates your work from ad-hoc data cleaning to systematic data engineering.
| Concept | This Lesson (Fundamentals) | Advanced Extension |
|---|---|---|
| Type assignment | Table.TransformColumnTypes with scalar types | Define a table type schema using type table [Col1 = number, Col2 = date, ...] and enforce it with Value.ReplaceType |
| Error handling | Table.ReplaceErrorValues and try...otherwise | Route error rows to a separate error log table using Table.SelectRowsWithErrors before cleaning, enabling audit trails |
| Culture handling | Single culture parameter per TransformColumnTypes call | Per-column culture via custom functions: parse column A as "en-US" date and column B as "de-DE" number in the same row |
| Reusability | Type steps embedded in individual queries | Shared type-enforcement functions deployed via Power BI Dataflows or parameterized M functions, acting as a centralized schema registry |
Fundamental vs. advanced type management patterns
As you progress to building enterprise-grade Power BI solutions, the concepts in this lesson form the bedrock on which advanced patterns are built. The M language's type system, while not as expressive as a full algebraic type system, is sufficiently powerful to enforce schemas, validate data contracts between sources and reports, and build resilient pipelines that degrade gracefully in the presence of malformed data. Mastering the fundamentals of type coercion, locale-aware parsing, and error handling is the prerequisite for all of these advanced patterns.
SECTION 9
Practice Problems
Question 1: CONCEPTUAL
Show answer
Table.TransformColumnTypes(Source, {{"Date", type date}}) parses dates differently depending on the execution environment. The solution is to always include the third culture argument (e.g., "en-US") to ensure locale-independent, deterministic parsing regardless of where the query executes.Question 2: BASIC CALCULATION
type number using the appropriate culture setting. What numeric values will these three strings produce?Show answer
Table.TransformColumnTypes(Source, {{"Price", type number}}, "de-DE"). Under the "de-DE" culture, the dot serves as the thousands separator and the comma serves as the decimal separator. Therefore: "2.499,99" → 2499.99, "850,00" → 850.00, "1.200.000,50" → 1200000.50. Without the "de-DE" culture parameter, the en-US default would interpret the first dot as a decimal point, producing either 2.49999 (incorrect) or an error due to the unexpected comma, depending on the exact parsing logic applied.Question 3: INTERMEDIATE
type text because neither format dominates in the first 200 rows. Describe a strategy using M to parse both formats correctly into a single type date column. Provide the M expression for a custom column that implements this strategy.Show answer
Table.AddColumn(Source, "ParsedDate", each if Text.Contains([EventDate], "-") then Date.FromText([EventDate], "en-US") else try Date.FromText([EventDate], "en-GB") otherwise null, type date). This works because ISO 8601 is parsed unambiguously by any locale (YYYY-MM-DD is universally recognized), while the DD/MM/YYYY format requires a UK or European locale. The try...otherwise null on the second branch catches any residual edge cases. After adding this column, remove the original "EventDate" text column and rename "ParsedDate" to "EventDate".Question 4: APPLIED
Table.Combine. The appended table has columns: Region (text), SaleDate (text), Amount (text). Design a Power Query strategy that correctly types SaleDate and Amount for all 12 sources, explaining where in the query chain you apply type changes and why.Show answer
TransformColumnTypes call impossible. The strategy: (1) Create a parameterized function fnLoadRegion that takes a file path and a culture string as inputs. Inside this function, load the CSV, apply Table.TransformColumnTypes(src, {{"SaleDate", type date}, {"Amount", type number}}, culture), and return the typed table. (2) Invoke this function with "en-US" for the six US files and "fr-FR" for the six French files. (3) Append all 12 typed result tables using Table.Combine. Because each source is parsed under its correct locale before combination, the resulting date and number columns are correctly typed and semantically consistent. This pattern is analogous to deserializing JSON with per-endpoint schema definitions in a microservices architecture.Question 5: CRITICAL THINKING
Show answer
try...otherwise null pattern implements manually. The current design pushes responsibility to the developer to opt into leniency, which is arguably the correct default for a tool that serves both exploratory and production use cases — better to surface errors early than to silently corrupt aggregations.SUMMARY
Summary
Summary
Power Query's type system supports a rich set of primitive types including number, text, date, datetime, datetimezone, logical, and duration. Every column must be explicitly typed before loading into the Vertipaq engine, and the primary mechanism for this is Table.TransformColumnTypes. While automatic type detection is convenient for prototyping, it relies on the system locale and a 200-row sample, making it unreliable for production use. The most critical best practice is to specify the culture parameter explicitly (e.g., "en-US", "de-DE") to ensure deterministic, environment-independent parsing of dates and numbers.
Parsing failures produce Error values — not nulls — which propagate destructively through downstream DAX calculations. Handle these using try...otherwise for per-value safety or Table.ReplaceErrorValues for bulk cleanup. The most dangerous bug is not an error but a silent misinterpretation — such as a date whose month and day are silently swapped due to a locale mismatch. Always clean sentinel values before casting, normalize non-standard boolean representations, and apply type changes with explicit culture before appending multi-source data. These disciplines transform ad-hoc data cleaning into reliable, reproducible data engineering.