Scheduled Refreshes — Schedule extract refreshes and manage failures conceptually

Historical Context & Motivation

Modern business intelligence platforms must serve dashboards whose underlying data changes constantly—transactional databases accumulate rows, cloud APIs emit new payloads, and data lakes ingest streaming events around the clock. In the early days of BI, analysts manually exported CSV files and re-imported them each morning, a tedious and error-prone ritual that left organizations working with stale information for hours or even days. Scheduled refreshes emerged as the answer to this operational bottleneck: a server-side mechanism that automatically re-queries source databases, rebuilds the optimized columnar snapshot known as a Tableau extract, and publishes fresh data to consumers without any manual intervention. Understanding how these schedules work—and how failures propagate—is essential knowledge for anyone deploying Tableau at scale.

The central question this lesson addresses is straightforward yet operationally critical: how do you design a reliable refresh strategy that keeps dashboards fresh, minimizes server load, and gracefully handles the inevitable failures that arise from network outages, credential expiration, and schema drift? We will explore the underlying mechanics, failure taxonomy, and best practices that govern this pipeline.

Core Principles & Definitions

Before diving into scheduling mechanics, it is important to ground ourselves in the foundational concepts that underpin extract-based analytics in Tableau. An extract is a compressed, columnar snapshot of a data source stored in the .hyper format. Unlike a live connection, which forwards every user query to the source database in real time, an extract is materialized on the Tableau server and queried locally by the Hyper engine. This architecture trades real-time freshness for dramatically faster query performance, reduced load on transactional databases, and offline availability. The scheduled refresh is the mechanism that keeps these materialized snapshots synchronized with the source of truth.

Visual Explanation — The Refresh Lifecycle

The lifecycle depicted above follows a well-defined sequence that mirrors a classic producer–consumer pipeline. When the clock reaches the scheduled time, the server enqueues a refresh task into the Backgrounder's internal job queue. The Backgrounder process dequeues the task, establishes a connection to the upstream data source using embedded or separately stored credentials, and executes the extract query. If the refresh type is full, a temporary .hyper file is built from scratch; if it is incremental, new rows are appended to the existing file. Upon successful completion, the server performs an atomic swap—replacing the old extract file with the new one in a single, indivisible operation so that dashboard users never see a partially built extract.

Critically, when any step in this pipeline fails, Tableau does not delete the existing extract. The old, now-stale snapshot remains in place and continues to serve queries. This design is analogous to a blue-green deployment strategy in software engineering: the new artifact must be fully healthy before the router switches traffic to it, and if the build fails, the old artifact remains live.

How Scheduling Works — Mechanisms & Configuration

Schedule Creation and Assignment

On Tableau Server, server administrators define named schedules at the server level using the Settings → Schedules page or the REST API. A schedule specifies a recurrence pattern (hourly, daily, weekly, monthly), a start time in the server's configured time zone, and an optional priority integer from 0 (highest) to 100 (lowest). When a content owner publishes or updates a workbook or data source with an extract, they assign it to one of these predefined schedules. Multiple extracts can share a schedule; the Backgrounder serializes or parallelizes their execution depending on the number of available Backgrounder worker threads.

On Tableau Cloud, the model is slightly different. Schedules are not site-wide shared objects; instead, each extract owner configures a refresh cadence directly on the published data source or workbook. The minimum interval on Tableau Cloud is 15 minutes for Tableau Bridge connections and hourly for cloud-native connectors. For sources behind a firewall, Tableau Bridge acts as a reverse-tunnel agent: it runs on an on-premises machine, polls the Tableau Cloud site for pending refresh tasks, and executes them locally, pushing the resulting .hyper file back to the cloud.

Priority and Concurrency

When multiple refresh tasks are enqueued simultaneously—a common scenario at midnight when all 'daily' schedules fire at once—the Backgrounder dequeues tasks in priority order. Within the same priority level, tasks are processed in FIFO order. The total concurrency is determined by the number of Backgrounder processes multiplied by the backgrounder.querylimit setting. If all threads are occupied, queued tasks wait, potentially delaying downstream dashboards. This is the classic thundering herd problem familiar from operating systems and distributed computing, and the remedy is the same: stagger your schedules.

REST API and Event-Driven Refreshes

Beyond time-based schedules, Tableau's REST API exposes the POST /api/{version}/sites/{siteId}/datasources/{datasourceId}/refresh endpoint, which allows external systems—CI/CD pipelines, Airflow DAGs, or dbt post-hooks—to trigger an extract refresh programmatically. This event-driven pattern decouples refresh timing from a fixed clock and instead ties it to data availability. When your upstream ETL writes a completion marker file, a webhook fires the Tableau refresh, guaranteeing that the dashboard updates within minutes of new data landing rather than waiting for the next scheduled window.

Failure Taxonomy & Alerting

Extract refresh failures are inevitable in production environments, and a robust governance strategy requires classifying failures by root cause so that the appropriate remediation can be applied. Tableau groups failures into several categories, each surfaced through the Background Tasks for Extracts administrative view, the Backgrounder log files (typically backgrounder-*.log), and the email alert system. Understanding this taxonomy helps administrators triage incidents efficiently.

Alerting and Suspension Policies

When a refresh task fails, Tableau Server sends an email notification to the content owner (and optionally to the site administrator). The email includes a truncated error message and a link to the administrative view for more details. On Tableau Server, administrators can configure a consecutive failure threshold via tsm configuration set -k backgrounder.failure_threshold_for_job_suspension. After the extract fails this many times in a row—the default is five—the task is automatically suspended, preventing it from consuming Backgrounder cycles on subsequent schedule runs. The owner receives a suspension notification and must manually resume the task after addressing the root cause.

Worked Example — Designing a Refresh Strategy

Consider a mid-sized e-commerce company that publishes three critical extracts to Tableau Server. The Sales Dashboard pulls from a 50-million-row PostgreSQL transactions table and must be updated nightly. The Inventory Tracker connects to a Snowflake warehouse and needs hourly updates. The Marketing Attribution workbook reads from a Google BigQuery dataset that is refreshed by an Airflow DAG at irregular intervals. The server has two Backgrounder processes, each running a single concurrent thread.

Full vs. Incremental vs. Live — Tradeoff Analysis

Connection to Advanced Topics — Governance at Scale

Scheduled refreshes are one component of a broader data governance strategy on the Tableau platform. As organizations scale beyond a few dozen extracts to hundreds or thousands, more sophisticated orchestration patterns become necessary. The concepts learned in this lesson serve as the foundation for several advanced capabilities that extend the refresh paradigm into enterprise-grade data operations.

As you advance in Tableau administration, you will encounter virtual connections and extract policies that consolidate refresh management for hundreds of workbooks into a single governed data asset. These features build directly upon the scheduling and failure-handling primitives covered in this lesson. Mastering the fundamentals of refresh cadence design, failure classification, and capacity estimation positions you to evaluate these advanced features with the right mental model—one grounded in systems thinking about caching, concurrency, and fault tolerance.

Practice Problems

Summary