Data Frames — Create and inspect data frames; understand columns vs rows - R Programming

SECTION 1

Historical Context & Motivation

Statistical computing has long grappled with a fundamental tension: real-world datasets contain columns of different types — numeric measurements alongside categorical labels, dates interleaved with character strings — yet the mathematical operations we perform on them demand structured, rectangular organization. Before the data frame emerged as a first-class construct, analysts were forced to maintain parallel arrays or use matrix representations that coerced all data to a single type, sacrificing type fidelity for structural convenience. The data frame solved this problem elegantly by wrapping a list of equal-length vectors into a table-like object, preserving each column's native type while enforcing the rectangular constraint that makes row-wise operations meaningful.

The intellectual lineage of the data frame traces back to the statistical concept of a case-by-variable matrix — a representation where each row is an observation and each column is a measured attribute. This convention, deeply rooted in experimental design and survey methodology, predates computing entirely. When John Chambers and his colleagues at Bell Labs began designing the S language in the 1970s, they sought to give this tabular abstraction a native programmatic representation, ultimately producing the data frame as we know it today.

1976 • S Language at Bell Labs

John Chambers and colleagues create the S language for statistical computing, introducing early concepts of structured data objects that would later evolve into data frames.

1988 • S3 Object System & data.frame

The S version 3 object system formalizes the data.frame class as a list of equal-length vectors with a row.names attribute, establishing the canonical tabular structure for statistical analysis.

1993 • R Project Begins

Ross Ihaka and Robert Gentleman begin developing R at the University of Auckland, inheriting the data frame construct from S and making it a cornerstone of R's data model.

2014 • dplyr and the Tidyverse

Hadley Wickham releases dplyr, introducing the tibble — a modern reimagination of the data frame with stricter defaults and cleaner printing — solidifying data frames as the central abstraction in R's data science ecosystem.

2019 • data.table Maturity

The data.table package reaches widespread adoption, offering a high-performance data frame variant with reference semantics, demonstrating the enduring importance of the tabular abstraction in large-scale data processing.

Understanding data frames is not merely a syntactic exercise — it addresses the fundamental question of how heterogeneous, real-world information can be organized into a structure that supports both programmatic manipulation and statistical reasoning. Every data science workflow in R, from importing a CSV to fitting a regression model, revolves around this single abstraction.

SECTION 2

Core Principles & Definitions

A data frame in R is formally a list of vectors (or factors) of equal length, augmented with a class attribute set to "data.frame" and a row.names attribute providing unique identifiers for each row. This dual nature — simultaneously a list and a table — is the key to understanding how R's subsetting and manipulation operators interact with data frames. Because it is a list, each column can hold a different atomic type; because the equal-length constraint is enforced, the structure can be indexed as a two-dimensional object with [row, col] notation.

Heterogeneous Columns

Each column is an independent vector that can be numeric, character, logical, or factor. Unlike matrices, no type coercion occurs across columns.

Equal-Length Constraint

All columns must have identical length, ensuring every row represents a complete observation. R will recycle shorter vectors if their length evenly divides the longest, but will error otherwise.

Row = Observation, Column = Variable

This convention — known as tidy data — maps directly to statistical modeling: each row is a case, each column is a measured or categorical attribute of that case.

Dual Indexing

Data frames support both list-style access (df$col, df[["col"]]) and matrix-style access (df[row, col]), giving programmers flexible extraction semantics.

Inspection Functions

R provides a rich set of introspection functions — str(), summary(), head(), dim() — that reveal the shape, types, and distributions within a data frame.

✦

KEY TAKEAWAY

Think of a data frame as a spreadsheet with type enforcement. In a spreadsheet, column A might hold names (strings) and column B might hold ages (numbers), and you expect every row to represent one person. A data frame encodes exactly this contract in R's type system: each column is a typed vector, each row is a record, and the runtime enforces that all columns agree on the number of records. If you've used SQL, the analogy is even tighter — a data frame is essentially an in-memory relational table without a schema definition language.

SECTION 3

Visual Explanation — Anatomy of a Data Frame

The top half shows the tabular view: rows are indexed by position ([1] through [4]), and each column header maps to a variable. The bottom half reveals the underlying list structure — four independently typed vectors bound together. Note how typeof() returns "list" while class() returns "data.frame" — a clear demonstration of S3 dispatch layered on top of a base type.

The diagram above captures a critical duality. When you print a data frame at the R console, you see a rectangular table with aligned columns and numbered rows — this is the tabular interface. But internally, R stores it as a named list where each element is a vector, and the row.names attribute tracks how those vectors are aligned. This is why df$name and df[[1]] both extract the first column as a plain vector — you are performing list extraction. Conversely, df[1, ] extracts a one-row data frame by pulling the first element from each list member, which is a fundamentally different operation that the [.data.frame method dispatches.

SECTION 4

How It Works — Creation and Inspection Mechanics

Creating Data Frames

The primary constructor is data.frame(), which accepts named arguments — each argument becomes a column. The function checks that all supplied vectors have the same length (or that shorter vectors can be recycled to match), constructs the internal list, and attaches the "data.frame" class. By default, R versions prior to 4.0 converted character vectors to factors via the stringsAsFactors parameter, a behavior that was changed to FALSE as the default in R 4.0.0 — a historically significant change you should be aware of when reading legacy code.

DATA FRAME CONSTRUCTOR

df <- data.frame(col₁ = vec₁, col₂ = vec₂, ..., colₖ = vecₖ)

Each vecᵢ is a vector of length n. The result is a data frame with n rows and k columns. If length(vecᵢ) differs across arguments, R applies recycling rules: shorter vectors are repeated if their length evenly divides n, otherwise an error is raised.

Inspection Functions

Function	Returns	Use Case
`str(df)`	Compact display of structure: column names, types, and first few values	First thing to call on any new data frame; gives a complete structural overview
`dim(df)`	Integer vector of length 2: c(nrow, ncol)	Quick shape check; equivalent to combining nrow() and ncol()
`head(df, n)`	First n rows (default 6)	Preview data without printing thousands of rows
`summary(df)`	Per-column summary statistics (min, median, mean, max for numeric; counts for factors)	Rapid distributional overview and NA detection
`names(df)`	Character vector of column names	Retrieve or set column names programmatically
`class(df)`	"data.frame" (character string)	Confirm the object's S3 class; useful in conditional logic
`sapply(df, class)`	Named character vector of each column's class	Verify column types in one call; essential before modeling

Essential data frame inspection functions in base R

Subsetting Semantics

The subsetting behavior of data frames is one of the most nuanced aspects of R. The single-bracket operator df[i, j] returns a data frame by default when selecting multiple columns, but drops to a vector when selecting a single column unless you specify drop = FALSE. The double-bracket df[[j]] always extracts a single column as a vector, mirroring its behavior on lists. The dollar operator df$col is syntactic sugar for df[["col"]] with partial matching enabled — a convenience that can introduce subtle bugs if column names share prefixes.

SECTION 5

Columns vs Rows — Deep Dive

The distinction between column-oriented and row-oriented operations is fundamental to writing efficient R code. Since each column is stored as a contiguous vector in memory, operations that apply a function to an entire column — such as computing a mean or performing vectorized arithmetic — exploit CPU cache locality and R's internal C loops. Row-oriented operations, by contrast, require assembling values from multiple list elements, which is inherently slower in R's memory model.

Left: column access extracts a single contiguous vector (highlighted in amber). Right: row access gathers one element from each of the k column vectors (highlighted in violet). The memory layout section shows why column-wise operations are inherently more efficient in R's column-major data frame storage.

An important subtlety illustrated above is the return type difference. When you extract a single column via df[, "gpa"], R returns a numeric vector — the data frame wrapper is dropped. When you extract a single row via df[2, ], R returns a one-row data frame because a row spans multiple types and cannot be represented as a single atomic vector. This asymmetry is a direct consequence of the column-major list-of-vectors implementation. Understanding this prevents a common class of bugs where functions expecting a data frame receive a vector, or vice versa.

⚠

Common Pitfall: apply() on Mixed-Type Frames

Calling apply(df, 1, function(row) ...) to iterate over rows coerces the entire data frame to a matrix first, which forces all columns to a single type (typically character). If you need row-wise operations on heterogeneous frames, use lapply(seq_len(nrow(df)), function(i) df[i, ]) or the tidyverse's purrr::pmap() to preserve column types.

SECTION 6

Worked Example — Building and Inspecting a Student Data Frame

Let us walk through the complete lifecycle of creating a data frame, inspecting its structure, and extracting information from it using both column-oriented and row-oriented access patterns.

Create, Inspect, and Query a Data Frame

1
Step 1 — Construct the data frame

We create a data frame representing student records with four columns of different types. The data.frame() constructor takes named arguments: students <- data.frame( name = c("Alice", "Bob", "Clara", "Dev", "Eve"), age = c(21L, 23L, 20L, 22L, 21L), gpa = c(3.80, 3.45, 3.92, 3.61, 3.78), enrolled = c(TRUE, FALSE, TRUE, TRUE, FALSE) ) Note the use of L suffix on integers to explicitly set the type rather than relying on R's default double precision.

A 5×4 data frame with columns: character, integer, numeric, logical.
2
Step 2 — Inspect the structure with str()

str(students) Output: 'data.frame': 5 obs. of 4 variables: $ name : chr "Alice" "Bob" "Clara" "Dev" ... $ age : int 21 23 20 22 21 $ gpa : num 3.8 3.45 3.92 3.61 3.78 $ enrolled: logi TRUE FALSE TRUE TRUE FALSE The str() output confirms that each column retained its intended type. The dollar-sign notation in the output mirrors the actual extraction syntax.

5 observations, 4 variables — types verified as chr, int, num, logi.
3
Step 3 — Check dimensions and preview

dim(students) # [1] 5 4 nrow(students) # [1] 5 ncol(students) # [1] 4 head(students, 3) # first 3 rows The dim() function returns a 2-element integer vector: rows first, columns second. This convention is consistent with matrix indexing in R where the first index always selects rows.

dim(students) → c(5, 4)
4
Step 4 — Extract a column (variable)

students$gpa # returns numeric vector: 3.80 3.45 3.92 3.61 3.78 students[["gpa"]] # identical result mean(students$gpa) # 3.712 Both $ and [[]] extract the column as a plain numeric vector, which we can immediately pass to mean() without any type conversion.

mean(students$gpa) = 3.712
5
Step 5 — Extract a row (observation) and filter

students[2, ] # Bob's complete record (1-row data frame) students[students$enrolled == TRUE, ] # all enrolled students The logical vector students$enrolled == TRUE produces c(TRUE, FALSE, TRUE, TRUE, FALSE), which when used as a row index selects rows 1, 3, and 4 — Alice, Clara, and Dev. This vectorized filtering is the idiomatic R approach to querying data frames.

Filtered data frame: 3 rows × 4 columns (Alice, Clara, Dev).
6
Step 6 — Add a new column

students$honors <- students$gpa >= 3.75 str(students) Output: 'data.frame': 5 obs. of 5 variables: $ name : chr "Alice" "Bob" "Clara" "Dev" ... $ age : int 21 23 20 22 21 $ gpa : num 3.8 3.45 3.92 3.61 3.78 $ enrolled: logi TRUE FALSE TRUE TRUE FALSE $ honors : logi TRUE FALSE TRUE FALSE TRUE Assigning to a new name via $ appends a column in place. The expression students$gpa >= 3.75 is evaluated element-wise, producing the logical vector c(TRUE, FALSE, TRUE, FALSE, TRUE) — indicating that Alice, Clara, and Eve meet the honors threshold. This new logical vector is bound as the fifth column, and the data frame grows from 5×4 to 5×5. No recycling is needed because the vector already has length 5, matching the existing rows.

students is now 5 × 5 with a new logical column 'honors' (TRUE for Alice, Clara, Eve; FALSE for Bob, Dev).

SECTION 7

Data Frame vs Other Structures

R offers several data structures for organizing collections of values, each with distinct trade-offs. Understanding when to use a data frame versus a matrix, list, or tibble is essential for writing idiomatic and performant R code. The table below provides a systematic comparison across the dimensions that matter most in practice.

Property	data.frame	matrix	list	tibble
Column types	Heterogeneous (each column independent)	Homogeneous (single atomic type)	Fully heterogeneous (no constraint)	Heterogeneous (same as data.frame)
Shape constraint	Rectangular (equal-length columns)	Rectangular (fixed nrow × ncol)	None (ragged structure allowed)	Rectangular (stricter than data.frame)
Subsetting drops dim?	Yes, single-column drops to vector	Yes, single-row/col drops to vector	N/A	No — always returns a tibble
Printing	Prints all rows (can flood console)	Prints all rows with dim header	Nested/recursive printing	Prints first 10 rows with types shown
Performance	Good for column ops; slower row ops	Fast for linear algebra; cache-friendly	Flexible but no vectorized ops	Same as data.frame (inherits class)
Use case	General tabular data; most modeling functions	Numerical computation; same-type grids	Nested or irregular structures	Tidyverse workflows; modern data science

Comparison of R's primary data structures

✦

WHEN TO USE WHAT

If your data is a grid of numbers destined for matrix algebra, use a matrix. If you need to store nested or ragged structures (like JSON-parsed API responses), use a list. For virtually everything else — CSV files, SQL query results, experimental data, model inputs — a data frame (or its modern cousin, the tibble) is the right choice. Think of the data frame as the relational table of R — it's the common currency of statistical functions, plotting libraries, and data pipelines.

SECTION 8

Connection to Advanced Concepts

The base R data frame, while powerful, has known limitations that have driven the development of two major alternatives in the R ecosystem. Understanding how the data frame relates to these advanced variants will prepare you for real-world data engineering tasks where performance and ergonomics matter.

Feature	Base data.frame	tibble (tidyverse)	data.table
Partial matching ($)	Enabled — df $n matches df$ name	Disabled — warns if no exact match	Disabled
stringsAsFactors	FALSE since R 4.0 (was TRUE)	Always FALSE	Always FALSE
Modification semantics	Copy-on-modify	Copy-on-modify	Modify-in-place (reference semantics)
Row grouping / joins	merge(), aggregate() — functional	dplyr verbs: group_by(), left_join()	dt[i, j, by] syntax — very fast
Speed at scale (10⁶+ rows)	Adequate for moderate sizes	Similar to base (with dplyr backend)	Highly optimized — often 10–100× faster
List-columns	Supported but awkward to create	First-class support via tidyr::nest()	Supported

Base data.frame vs. modern alternatives

As you progress to courses in database systems, machine learning, or large-scale data processing, you will encounter these alternatives frequently. The tibble is the default for tidyverse workflows and offers sensible guardrails (no partial matching, cleaner printing). The data.table excels in scenarios involving millions of rows, where its in-place modification and optimized grouping algorithms provide dramatic speedups. Crucially, both tibbles and data.tables inherit from data.frame, meaning any function that accepts a data frame will accept either variant — the base data frame is truly the foundational abstraction upon which the modern ecosystem is built.

ℹ

Looking Ahead

Once you are comfortable with creating and inspecting data frames, the next steps are: (1) reading external data into frames with read.csv() and readr::read_csv(), (2) reshaping between wide and long formats with tidyr::pivot_longer(), and (3) transforming and summarizing with dplyr verbs. Each of these builds directly on the column-as-variable, row-as-observation paradigm you have mastered here.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

A data frame in R is said to be "a list of equal-length vectors." Explain why typeof(df) returns "list" while class(df) returns "data.frame". How does this distinction between type and class relate to R's S3 object system?

Show answer

In R's S3 object system, typeof() interrogates the underlying C-level storage representation (SEXP type), which for a data frame is a generic list (VECSXP). The class() function reads the class attribute, which controls method dispatch. When you call print(df), R dispatches to print.data.frame based on the class attribute, producing tabular output rather than the recursive list printing you'd get from a plain list. This layered design — base type for storage, class attribute for behavior — is the hallmark of S3 dispatch and explains why a data frame behaves like a table while being stored as a list.

Question 2: BASIC CALCULATION

Given the following code:

df <- data.frame(
  product = c("A", "B", "C", "D"),
  price   = c(10.5, 23.0, 7.25, 15.75),
  qty     = c(100L, 50L, 200L, 75L)
)

What are the outputs of dim(df), sapply(df, class), and sum(df$price * df$qty)?

Show answer

dim(df) returns c(4, 3) — 4 rows and 3 columns. sapply(df, class) returns a named character vector: product = "character", price = "numeric", qty = "integer". For the revenue calculation, df$price * df$qty performs element-wise multiplication yielding c(1050, 1150, 1450, 1181.25), and sum() of that vector is 4831.25. Note that multiplying an integer vector by a numeric vector results in a numeric vector via type promotion.

Question 3: INTERMEDIATE

Consider this code:

df <- data.frame(x = 1:4, y = c("a", "b", "c", "d"))
result1 <- df[, 1]
result2 <- df[, 1, drop = FALSE]
result3 <- df[1, ]

For each of result1, result2, and result3, state (a) the class of the result, (b) the value of is.data.frame() on it, and (c) why the behavior differs.

Show answer

result1 is an integer vector (class "integer", is.data.frame returns FALSE) because selecting a single column with [, j] drops the data frame wrapper by default. result2 is a 4×1 data frame (is.data.frame returns TRUE) because drop = FALSE preserves dimensionality, analogous to numpy's keepdims parameter. result3 is a 1×2 data frame (is.data.frame returns TRUE) because row extraction cannot coerce heterogeneous types (integer and character) into a single vector, so R must preserve the data frame container. This asymmetry between row and column extraction is a direct consequence of the list-of-typed-vectors implementation.

Question 4: APPLIED

You receive a CSV file of server log data with columns: timestamp (character), response_ms (numeric), status_code (integer), and endpoint (character). Write R code that: (1) reads it into a data frame, (2) verifies the column types, (3) filters to only 500-series errors, and (4) computes the mean response time of those errors. Use base R only.

Show answer

A complete solution:

logs <- read.csv("server_logs.csv")
str(logs)  # verify types
errors_5xx <- logs[logs$status_code >= 500 & logs$status_code < 600, ]
mean_error_response <- mean(errors_5xx$response_ms, na.rm = TRUE)

The str() call is essential to verify that response_ms was not parsed as character (which can happen if the CSV contains non-numeric entries). The filtering uses a compound logical expression; note the use of & (vectorized AND) rather than && (scalar AND). The na.rm = TRUE argument protects against NA values that could propagate through the mean calculation.

Question 5: CRITICAL THINKING

R's data frame stores data in column-major order (each column is a contiguous vector), while row-oriented databases like traditional OLTP systems store data row by row. Discuss how this storage layout affects the performance of (a) computing the average of a single numeric column across all rows, (b) retrieving all fields for a single observation, and (c) appending a new observation. Under what workload patterns would you consider converting a data frame to a matrix, and what trade-off would you accept?

Show answer

(a) Column-major storage is ideal for column aggregation: mean(df$x) iterates over a contiguous memory region, maximizing cache line utilization and enabling SIMD vectorization. This is O(n) with excellent constant factors. (b) Retrieving a single row requires indexing into k separate vectors at position i, which involves k pointer dereferences to potentially non-adjacent memory regions — slower than a row-major layout where all fields would be contiguous. (c) Appending a row is expensive because R uses copy-on-modify semantics: each of the k column vectors must be reallocated and copied to accommodate one additional element, resulting in O(n × k) work. Pre-allocating with a known size or collecting rows in a list and calling do.call(rbind, list_of_rows) at the end is the idiomatic workaround. You would convert to a matrix when all columns share a single numeric type and you need linear algebra operations (matrix multiplication, eigendecomposition), accepting the trade-off of losing type heterogeneity in exchange for R's optimized LAPACK-backed matrix routines.

SUMMARY

Lesson Summary

Summary

The R data frame is a list of equal-length typed vectors that presents a tabular interface for working with heterogeneous datasets. You create data frames with data.frame() by supplying named vectors of equal length, and inspect them with str(), dim(), head(), summary(), and names(). Each column represents a variable and can hold its own atomic type (numeric, character, logical, factor), while each row represents a single observation across all variables.

Column extraction via $ or [[]] returns a plain vector, enabling efficient vectorized operations that exploit R's column-major storage layout. Row extraction via [i, ] always returns a data frame because it spans heterogeneous types. Understanding this asymmetry between row and column subsetting is essential for avoiding type-related bugs. The base data frame serves as the foundation for modern variants like tibbles and data.tables, making it the most important data structure to master in R.