Data Visualization
Direct plotting with Pandas, advanced charting with Matplotlib, and distribution/bivariate visualization with Seaborn.
Topics in this chapter
- Direct Plotting: Line Chart, Bar Chart, Pie Chart, Scatter Plot, Box Plot, Histogram
- Matplotlib: Line, Bar, Histogram, Scatter, Stack Plot, Pie Chart, Heatmap
- Using text() and annotate() methods to add text in charts
- Creating multiple charts in same plot
- Seaborn: Strip Plot, Box Plot, Swarm Plot, Joint Plot
Direct Plotting Concepts and Standard Statistical Chart Types
Direct Plotting: Foundations and Principles
Direct plotting refers to visualization routines invoked as methods bound directly to Pandas DataFrame and Series objects. Formally, if denotes a data frame with observations and variables, direct plotting exposes a family of mappings , where indexes the chart type (line, bar, scatter, etc.) and is the space of rendered graphical objects. In Pandas, each is implemented as a thin wrapper around Matplotlib's pyplot primitives, meaning the analyst retains access to the full Matplotlib object model (axes, figures, legends, tick formatters) while enjoying dramatically reduced syntactic overhead for exploratory work.
The theoretical justification for maintaining a repertoire of standard chart types rests on the perceptual hierarchy established by Cleveland and Mc, who ranked graphical encodings by the accuracy with which human observers can decode quantitative information. From most to least accurate: position along a common scale, length, angle, area, and finally color saturation or hue. A mature visualization discipline therefore privileges charts that encode the target variable through position or length (line plots, bar plots, scatter plots) and reserves angle- or area-based encodings (pie charts, bubble charts) for contexts where approximate part-to-whole intuition is sufficient. Edward Tufte's complementary principle of the data-ink ratio — the fraction of a graphic's ink devoted to the non-redundant display of data — further counsels against chartjunk: decorative gradients, gratuitous 3-D effects, and redundant gridlines all subtract from analytical clarity.
The Line Plot: Temporal and Functional Relationships
A line plot represents a (typically univariate) response as a piecewise-linear function of an ordered index , most often time. Given observations with , the rendered curve is the union of line segments . Line plots are the canonical choice for time series data — GDP growth, stock prices, monthly sales — because the connected segments encode the assumption of continuity and invite the reader to interpret local slopes as rates of change. In economic applications, a steepening curve signals accelerating change; a decelerating curve signals moderation.
A subtle but powerful design principle is banking to 45°: the aspect ratio of the plotting frame should be chosen so that the median absolute slope of the line segments is approximately unity. When slopes cluster near , the human visual system is maximally sensitive to changes in rate. In Pandas, one invokes a line plot via df.plot(kind='line') or df.plot.line(). Multiple columns are automatically rendered as superposed series sharing a common axis — a convenience that demands caution when the series differ by orders of magnitude, since a high-variance series compresses a low-variance one into an unreadable flat line. The remedy is either rescaling (e.g., indexing each series to 100 at a base period) or employing a secondary -axis via Matplotlib's twinx().
Design considerations: The x-axis must be uniformly scaled; treating disjoint categories as a continuous sequence distorts perceived rates of change. Best practices advocate for direct labeling of curves rather than relying on separate legends, and for limiting the number of simultaneously displayed lines to maintain clarity. When the x-axis is time, Matplotlib provides dates locators and formatters to handle datetime objects natively.
The Bar Plot: Categorical Comparison
A bar plot encodes a quantitative magnitude as the length of a rectangle extending from a fixed baseline, typically zero, along a categorical axis. If denotes the value associated with category , the -th bar occupies the region for bar width . Because length is the second-most-accurate perceptual channel after position, bar plots are the workhorse for cross-sectional comparison: revenue by product line, unemployment by region, market share by firm.
The zero-baseline rule is non-negotiable. Truncating the -axis exaggerates differences: two bars of heights 101 and 100, plotted on an axis beginning at 99, appear to differ by a factor of two — a visual lie that has no analogue in a line plot where the reader interprets position rather than length.
Variants include:
- Grouped bar plot: Side-by-side bars for multiple series within each category, useful for comparing male and female wages across occupations.
- Stacked bar plot: Bars partitioned into sub-segments, useful for displaying composition within totals, such as the energy mix within each country's generation. The critical limitation is that stacked bars lack a common baseline for all but the bottom segment, complicating relative comparisons.
- Horizontal bars (
barh): Preferred when category labels are long, since they permit left-aligned text read naturally.
Bars should be ordered meaningfully — often descending, unless a natural categorical order exists. Decorations such as 3D effects, shadows, or excessive grid lines reduce the data-ink ratio and should be eschewed.
The Pie Chart: Part-to-Whole Composition
A pie chart partitions a circle into sectors whose central angles represent shares of a whole. Although ubiquitous in business dashboards, pie charts are widely deprecated in the visualization literature because angle and area — the encodings they rely upon — sit low on the Cleveland–McGill hierarchy. Empirical studies show that observers systematically misjudge sectors near 90° and 180°, and that comparison across multiple pies is substantially less accurate than comparison across stacked bars.
The pie chart is defensible only in narrow circumstances: when the number of categories is small (three to five), when one or two categories dominate and the analytical message is "this component is the overwhelming majority," and when precise quantitative comparison is not the goal. In all other cases, a stacked bar or a treemap conveys the same part-to-whole information with higher perceptual fidelity. If a pie must be used, best practice dictates sorting slices clockwise from the 12-o'clock position in descending order of magnitude and labeling each sector with both the category name and percentage.
The Scatter Plot: Bivariate Association
A scatter plot displays observations as points in the Cartesian plane and is the primary instrument for diagnosing bivariate relationships. Its power lies in revealing not only the direction and strength of association but also its functional form (linear, logarithmic, quadratic), heteroskedasticity (variance of changing with ), and the presence of outliers or clusters that summary statistics conceal.
The canonical cautionary tale is Anscombe's quartet: four datasets with identical means, variances, Pearson correlation , and ordinary-least-squares regression lines, yet radically different scatter-plot appearances — one linear, one curved, one dominated by a single leverage point, and one with a vertical stack plus an outlier. The quartet demonstrates that the correlation coefficient is a necessary but insufficient summary; the scatter plot is the sufficient one.
In economic analytics, scatter plots underpin demand estimation (price versus quantity), production-function fitting (capital and labor versus output), and visual inspection of residuals against fitted values. Enhancements include overlaying a regression line, encoding a third variable through point color or size (producing a bubble plot), and using transparency (alpha < 1) to mitigate overplotting in large samples. A complementary smooth trend line computed via local regression (LOESS) or a parametric model can be superimposed to summarize the relationship. Never draw connecting lines between points in a scatter plot unless the x-axis is truly ordered.
The Box Plot: Distributional Summary
A box plot (box-and-whisker plot), introduced by , compresses a univariate distribution into a five-number summary: the minimum, first quartile , median , third quartile , and maximum, augmented by an explicit rule for flagging outliers. The interquartile range is , and Tukey's fences define the whisker endpoints as the most extreme observations lying within . Points beyond these fences are plotted individually as outliers.
The factor 1.5 is not arbitrary: for a Gaussian distribution, approximately 99.3% of observations fall within these fences, so the rule flags roughly 0.7% of a normal sample — close to the conventional significance threshold — while remaining robust to moderate departures from normality. The box plot conveys central tendency, spread, skewness (asymmetry visible in the median's position within the box), and the presence of extreme observations, all in a format that is robust to outliers.
Box plots are most effective when comparing distributions across multiple categories side by side, for instance salary distributions by academic rank. Limitations include their inability to reveal multimodality or fine-grained distributional shape; a single box will hide bimodality. For small sample sizes (), the five-number summary is unstable and misleading. As a complement or alternative, violin plots (combining a kernel density estimate with a box plot) can show the full distribution shape.
The Histogram: Density Estimation
A histogram partitions a continuous variable into adjacent, non-overlapping bins of width and displays counts or densities as bar heights. Mathematically, the histogram estimates the probability density function of the underlying random variable. For a sample of size , the histogram density at bin is:
When and such that , the histogram converges in probability to the true density. This makes the histogram a simple, non-parametric density estimator with a uniform kernel.
Use cases center on assessing distributional characteristics: modality, symmetry, heavy tails, and gaps. Overlaying a histogram with a theoretical density curve (e.g., normal) allows swift diagnostics of normality assumptions. However, the visual appearance is highly sensitive to bin width selection. Rules of thumb include Sturges' formula , Scott's normal reference rule , and Freedman-Diaconis rule . The analyst should experiment with multiple bin widths to avoid visual artifacts.
Matplotlib Plotting Techniques and Advanced Chart Customization
Matplotlib serves as the foundational visualization library in Python's data science ecosystem, providing a comprehensive object-oriented API for creating publication-quality graphics. At its core, Matplotlib operates on a hierarchical structure where a Figure object contains one or more Axes objects, each representing an individual plotting area with its own coordinate system, data limits, and visual elements. Understanding this architecture is essential for advanced customization and complex multi-panel visualizations.
Line Plots and Time Series Visualization
Line plots represent the most fundamental visualization technique, connecting discrete data points with continuous line segments to reveal temporal trends and sequential patterns. The basic call is ax.plot(x, y, fmt), where fmt encodes color, marker, and line style (e.g., 'r--' for red dashed line). For multiple series, repeated calls layer additional lines on the same axes.
The appearance of each line is governed by keyword arguments: color, linestyle, linewidth, marker, markersize, and alpha (transparency). A typical graduate-level plot would set these explicitly for publication quality:
import matplotlib.pyplot as plt
import numpy as np
# Economic time series: GDP growth rates
quarters = np.arange(2020, 2026, 0.25)
gdp_growth = 2.5 + 1.5 * np.sin(2 * np.pi * quarters / 4) + np.random.normal(0, 0.3, len(quarters))
fig, ax = plt.subplots(figsize=(12, 6))
ax.plot(quarters, gdp_growth, 'b-', linewidth=2, label='GDP Growth Rate')
ax.axhline(y=0, color='r', linestyle='--', alpha=0.5, label='Zero Growth')
ax.fill_between(quarters, gdp_growth, 0, where=(gdp_growth > 0),
alpha=0.3, color='green', label='Positive Growth')
ax.fill_between(quarters, gdp_growth, 0, where=(gdp_growth < 0),
alpha=0.3, color='red', label='Negative Growth')
ax.set_xlabel('Time (Quarters)', fontsize=12)
ax.set_ylabel('GDP Growth Rate (%)', fontsize=12)
ax.set_title('Quarterly GDP Growth with Business Cycle Indicators', fontsize=14)
ax.legend(loc='upper right')
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
Axes objects expose set_xlim, set_ylim, and tick formatting routines that allow precise control over the displayed range. For time-series data, Matplotlib provides dates locators and formatters to handle datetime objects. The economic intuition behind line plots lies in their ability to reveal business cycles, secular trends, and structural breaks in time series data.
Bar Plots for Categorical Comparison
For a dataset with categories and corresponding values , bar plots map each to a bar height. The function ax.bar(x, height, width=0.8, ...) draws the bars. Important customizations include width to control bar thickness, color for uniform fill or a list of colors per bar, edgecolor and linewidth for visibility, and yerr for error bars.
Grouped and stacked bar charts require careful alignment of bar positions. For grouped bars with groups and measurements per group, one computes shifted coordinates using numpy.arange and offsets:
# Market share analysis across sectors
sectors = ['Technology', 'Healthcare', 'Finance', 'Energy', 'Consumer']
market_share_2024 = [28.5, 18.2, 15.8, 12.3, 25.2]
market_share_2025 = [32.1, 19.5, 14.2, 10.8, 23.4]
x = np.arange(len(sectors))
width = 0.35
fig, ax = plt.subplots(figsize=(10, 6))
bars1 = ax.bar(x - width/2, market_share_2024, width, label='2024', color='steelblue')
bars2 = ax.bar(x + width/2, market_share_2025, width, label='2025', color='coral')
ax.set_xlabel('Market Sectors', fontsize=12)
ax.set_ylabel('Market Share (%)', fontsize=12)
ax.set_title('Sectoral Market Share Comparison: 2024 vs 2025', fontsize=14)
ax.set_xticks(x)
ax.set_xticklabels(sectors)
ax.legend()
# Add value labels on bars
for bars in [bars1, bars2]:
for bar in bars:
height = bar.get_height()
ax.annotate(f'{height:.1f}%',
xy=(bar.get_x() + bar.get_width() / 2, height),
xytext=(0, 3), textcoords="offset points",
ha='center', va='bottom', fontsize=9)
plt.tight_layout()
plt.show()
Stacked bars are drawn by specifying the bottom argument, which accumulates the heights of previous layers:
p1 = ax.bar(ind, men_means, width, yerr=men_std, label='Men')
p2 = ax.bar(ind, women_means, width, bottom=men_means, yerr=women_std, label='Women')
Scatter Plots and Correlation Analysis
Matplotlib's ax.scatter(x, y, s=None, c=None, marker='o', ...) maps additional dimensions to marker size (s) and color (c). For data analysis, mapping a third continuous variable to color via a colormap is standard:
# Investment vs Returns analysis
np.random.seed(42)
investment = np.random.uniform(10, 100, 50)
returns = 0.8 * investment + np.random.normal(0, 15, 50)
risk_level = np.random.choice(['Low', 'Medium', 'High'], 50)
fig, ax = plt.subplots(figsize=(10, 7))
colors = {'Low': 'green', 'Medium': 'orange', 'High': 'red'}
for risk in ['Low', 'Medium', 'High']:
mask = risk_level == risk
ax.scatter(investment[mask], returns[mask],
c=colors[risk], label=f'{risk} Risk',
alpha=0.6, s=80, edgecolors='black', linewidth=0.5)
# Add trend line
z = np.polyfit(investment, returns, 1)
p = np.poly1d(z)
ax.plot(np.sort(investment), p(np.sort(investment)), "r--", alpha=0.8, linewidth=2,
label=f'Trend: y={z[0]:.2f}x+{z[1]:.2f}')
ax.set_xlabel('Investment (Million $)', fontsize=12)
ax.set_ylabel('Returns (Million $)', fontsize=12)
ax.set_title('Investment-Return Relationship by Risk Profile', fontsize=14)
ax.legend()
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
The s parameter is the marker area in points squared, so area is proportional to , not linear in the variable. To make marker areas proportional to a quantity , set .
Heatmaps for Matrix Visualization
Heatmaps visualize matrix-structured data by mapping each element to a color in a two-dimensional grid. Matplotlib offers ax.imshow(Z, ...) for image-like display and ax.pcolormesh(X, Y, Z, ...) for irregular grids. For a regular grid like a correlation matrix:
import seaborn as sns
# Correlation matrix of economic indicators
indicators = ['GDP', 'Unemployment', 'Inflation', 'Interest Rate', 'Stock Market']
correlation_matrix = np.array([
[1.00, -0.65, 0.45, 0.30, 0.75],
[-0.65, 1.00, -0.20, -0.15, -0.55],
[0.45, -0.20, 1.00, 0.80, 0.35],
[0.30, -0.15, 0.80, 1.00, 0.25],
[0.75, -0.55, 0.35, 0.25, 1.00]
])
fig, ax = plt.subplots(figsize=(8, 6))
im = ax.imshow(correlation_matrix, cmap='coolwarm', vmin=-1, vmax=1, aspect='equal')
fig.colorbar(im, ax=ax, shrink=0.8)
ax.set_xticks(np.arange(len(indicators)))
ax.set_yticks(np.arange(len(indicators)))
ax.set_xticklabels(indicators, rotation=45, ha='right')
ax.set_yticklabels(indicators)
ax.set_title('Correlation Matrix of Economic Indicators', fontsize=14)
plt.tight_layout()
plt.show()
The origin parameter controls the placement of the first row, and aspect='equal' forces square cells. Adding text inside cells with ax.text(j, i, f'{z:.2f}', ha='center', va='center') yields an annotated heatmap.
Annotations and Mathematical Expressions
Matplotlib annotates figures with descriptive text at arbitrary positions using either ax.text(x, y, string) or ax.annotate(string, xy=(x, y), xytext=(dx, dy), arrowprops=dict(...)). The coordinate system can be data coordinates, axes fraction (0 to 1), or figure fraction by setting the transform argument. Crucially, Matplotlib renders LaTeX mathematical expressions enclosed in $...$ when the text string contains math delimiters. This enables in-figure equations such as or .
ax.text(0.5, 0.95, r'$\frac{\partial \mathcal{L}}{\partial \theta} = 0$',
transform=ax.transAxes, fontsize=14, ha='center')
ax.annotate('Maximum', xy=(x_max, y_max), xytext=(x_max+1, y_max+2),
arrowprops=dict(arrowstyle='->', color='black'))
Organizing Multiple Subplots
Complex analytical narratives often require multiple views of the same data or side-by-side comparisons. Matplotlib's plt.subplots(nrows, ncols, figsize=(w,h), sharex=False, sharey=False) creates a regular grid of Axes objects:
fig, axes = plt.subplots(2, 2, figsize=(10, 8), sharex='col', sharey='row')
For non-uniform arrangements, GridSpec objects (via fig.add_gridspec(...)) permit different row and column heights/widths and spanning across multiple cells. The tight_layout() method automatically adjusts spacing, or subplots_adjust can manually set hspace, wspace. When combining different plot types, a secondary y-axis can be added via ax.twinx() to overlay two different scales — a common technique for financial time series displaying both price and volume.
Visualizing Distributions and Bivariate Relationships Using Seaborn
Seaborn provides a suite of high-level plotting functions that integrate statistical computation with aesthetically refined defaults, enabling analysts to move beyond simple summaries and uncover underlying data structures. We focus on advanced techniques for visualizing univariate distributions using strip plots, box plots, and swarm plots, and for examining bivariate relationships through joint plots.
Univariate Distribution Visualization
Strip Plots and the Problem of Overplotting
A strip plot displays each observation as a point along a single categorical or quantitative axis. Formally, given observations , a naïve strip plot maps each to the coordinate for some constant . When is large or the support of is discrete, points collapse onto one another, producing overplotting — a visual pathology that obscures the empirical density .
The standard remedy is jittering, which perturbs each point by an additive noise term:
where controls the vertical spread. The choice of involves a bias–variance trade-off: too small preserves overplotting, too large introduces spurious apparent variation. A useful heuristic is to set proportional to the inverse square root of the local density.
import seaborn as sns
tips = sns.load_dataset('tips')
sns.stripplot(x='day', y='total_bill', data=tips, jitter=True, alpha=0.6)
The economic intuition is straightforward: a strip plot preserves every datum, making it the most faithful representation of the sample. Unlike aggregated summaries, it allows the analyst to spot multimodality, gaps, and clusters that parametric summaries would conceal. The hue parameter can further stratify the data by a second categorical variable, and dodge=True separates the hue groups to avoid superposition.
Box Plots and Tukey's Fences
The box plot, introduced by John Tukey, compresses a distribution into a five-number summary. Seaborn's boxplot function computes these statistics and draws a box bounded by and , with a line at the median. The whiskers extend to the most extreme data point within of the quartile. Points beyond the whiskers are plotted individually as outliers:
sns.boxplot(x='day', y='total_bill', data=tips, hue='time', whis=1.5, showfliers=True)
The box plot's principal virtue is comparability: placing boxes side by side across categories (e.g., days of the week) lets the analyst read off location shifts (median differences), scale shifts (IQR differences), and skewness (asymmetry of whiskers) at a glance. Its limitation is that it imposes a unimodal, roughly symmetric template on the data; a bimodal distribution can produce a box plot indistinguishable from a unimodal one. Always pair a box plot with a strip plot or swarm plot to verify whether the summary statistics mask interesting features.
Swarm Plots and the Beeswarm Algorithm
The swarm plot resolves the tension between the strip plot's fidelity and its overplotting by enforcing a non-overlap constraint. The beeswarm algorithm positions each point at such that the Euclidean distance between any two points exceeds the marker diameter :
Points are placed sequentially, each assigned the smallest satisfying the constraint. The resulting silhouette approximates a rotated kernel density estimate, giving the swarm plot the interpretive richness of a violin plot while retaining individual observations:
sns.swarmplot(x='day', y='total_bill', data=tips, size=4, hue='time', dodge=True)
The computational cost of the beeswarm algorithm is in the worst case, which makes swarm plots impractical for . For larger samples, revert to jittered strip plots with reduced alpha or to aggregated representations such as violin plots. When combined with a box or violin plot (e.g., drawing the swarm on top of a box plot), one obtains both the summary statistics and the underlying data pattern.
Visualizing Bivariate Relationships with Joint Plots
Seaborn's joint plot (generated by jointplot) creates a multi-panel figure that combines a central bivariate plot with marginal univariate distributions along the axes. This integrated view is invaluable for simultaneously assessing the relationship between two variables and the distribution of each one. The function supports several central plot kinds through the kind parameter:
with sns.axes_style('white'):
sns.jointplot(data=tips, x='total_bill', y='tip', kind='scatter')
- Scatter (
kind='scatter'): Shows raw data points, ideal for detecting linear or non-linear associations, clusters, and bivariate outliers. The marginal histograms (default) or KDEs reveal the marginal distributions of and . - Regression (
kind='reg'): Overlays a linear regression line with a confidence interval, computed viascipy.stats.linregress. This adds an inferential layer, allowing the analyst to quickly gauge the direction and strength of a linear relationship. - Residual (
kind='resid'): Plots the residuals against , with a horizontal line at zero. This is a diagnostic tool for regression assumptions: uniform scatter around zero suggests homoscedasticity; funnel shapes indicate heteroscedasticity. - Kernel Density Estimate (
kind='kde'): Draws contour lines of the bivariate kernel density estimate . The contours represent regions of equal probability density. Bandwidth selection is critical; under-smoothing creates spurious modes, while over-smoothing obscures structure. - Hexagonal binning (
kind='hex'): For large datasets where scatter plots become overplotted, this plot divides the plane into hexagonal bins and colors each bin according to the count of points inside. Hexagonal tiling offers two advantages over square bins: each hexagon has six equidistant neighbors rather than four, yielding more isotropic neighborhood structure; and for a given bin area, hexagons approximate circles more closely, reducing directional bias in density estimation.
The jointplot function returns a JointGrid object, which can be further customized by accessing its components: g.ax_joint (central axes), g.ax_marg_x, g.ax_marg_y:
g = sns.jointplot(data=tips, x='total_bill', y='tip', kind='hex')
g.ax_marg_x.set_ylim(0, 20)
g.ax_joint.axhline(y=3, color='r')
A crucial parameter is ratio (default 5), which controls the height ratio of the marginal axes to the joint axis. The space parameter adjusts the gap between the joint and marginal plots. When interpreting a joint plot, the analyst should simultaneously notice patterns in the scatter/hex/kde and the marginal distributions. For example, in the tips dataset, the marginal distribution of total_bill is right-skewed, while tip is also skewed but less so; the bivariate scatter shows a positive correlation with increasing variance (heteroscedasticity).
Aesthetic Refinement for Professional Analytical Reporting
Seaborn's theming system enables quick adjustments that elevate a plot from exploratory graphic to presentation-ready figure. The sns.set_style() function accepts 'darkgrid', 'whitegrid', 'dark', 'white', and 'ticks'. The sns.set_context() adjusts scale for 'paper', 'notebook', 'talk', or 'poster':
sns.set_style('whitegrid')
sns.set_context('talk')
Beyond cosmetics, three interpretive principles should guide the graduate analyst:
- Always interrogate the marginals. A strong bivariate correlation can coexist with pathological marginals (e.g., heavy tails) that invalidate downstream assumptions.
- Prefer raw data over abstractions. Strip and swarm plots reveal structure that box plots and KDEs smooth away.
- Use color semantically. The
hueparameter should encode a meaningful grouping variable, not merely decorate.