Matplotlib, Seaborn and Plotting Tools

In this issue, we will introduce mainly the Matplotlib library for plotting, along with its good friend Seaborn as planned. While compiling this issue, I improved the layout system of the entire website's posts, hope you like it. I will also change some styles in this issue.

Previously, we have looked at Numpy and Pandas, and we can basically obtain and process the data. The next direct question is: what do these data actually look like? At this point, just looking at a bunch of numbers can really be a bit dizzying, so we need to plot.

In fact, Matplotlib is still very important for ML. Plotting is not just for aesthetics; many times, whether a model is overfitting, whether the data has strange distributions, or whether two variables are related can be much clearer at a glance in a graph than in a table.

Of course, Matplotlib can plot a lot of things. In this issue, we will first discuss the most commonly used items, and then we can gradually fill in the gaps when writing our own projects.

1. First, prepare Matplotlib

If you don't have matplotlib in your environment, you can install it in the terminal or Colab:

pip install matplotlib seaborn

Then we generally import it like this. Here, pyplot is basically our most commonly used plotting tool, and everyone calls it plt, so don't be nervous when you see plt later.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

2. The most basic line graph

Let's start with the simplest line graph. Suppose we have a sin curve. Here, linspace will help us take 200 numbers between -10 and 10, and then we can just put it into np.sin() to get started.

x = np.linspace(-10, 10, 200)
y = np.sin(x)

plt.figure(figsize=(9, 5))
plt.plot(x, y, color="royalblue", linewidth=2, label="sin(x)")
plt.axhline(0, color="gray", linewidth=0.8)
plt.title("A simple sine curve")
plt.xlabel("x")
plt.ylabel("sin(x)")
plt.grid(alpha=0.25)
plt.legend()
plt.show()

After running it, you will see a very standard curve.figure is the entire canvas,plot is to draw lines,xlabel and ylabel are the names of the axes, and the final show() is to actually display the graph. In fact, none of these lines are strictly necessary, but adding title, label, and grid makes it much clearer. A graph that you can understand is also easier for others to understand.

3. Scatter plot: to see if there is any relationship between two things

I think scatter plots are very common in ML. For example, we want to see if there is any relationship between study hours and score. To make the example runnable directly, I will use random data to simulate it.

rng = np.random.default_rng(7)
study_hours = rng.uniform(1, 10, 90)
score = 42 + study_hours * 5.4 + rng.normal(0, 7, 90)

plt.figure(figsize=(8, 5))
plt.scatter(study_hours, score, color="#ff7a59", alpha=0.72)
plt.title("Study hours and score")
plt.xlabel("Study hours")
plt.ylabel("Score")
plt.grid(alpha=0.2)
plt.show()

In this graph, each dot represents a piece of data.alpha is the transparency; if there are many dots, adding a bit of transparency will make it easier to see where it is particularly dense.

This example will likely show an upward trend, but that doesn't mean studying longer will definitely lead to a higher score; it just appears that there is a correlation in this dataset. It's important to distinguish this; correlation does not imply causation, and caution is still needed when doing data analysis later.

4. Bar chart and histogram

Bar charts are suitable for comparing different categories, such as the accuracy of different models. Histograms, on the other hand, show the distribution of a number. These two types of graphs may seem basic, but they are really used in model evaluation.

models = ["Logistic Regression", "Decision Tree", "Random Forest"]
accuracy = [0.79, 0.83, 0.89]

plt.figure(figsize=(9, 5))
plt.bar(models, accuracy, color=["#90caf9", "#81c784", "#ffcc80"])
plt.ylim(0, 1)
plt.title("Model accuracy comparison")
plt.ylabel("Accuracy")
plt.show()

values = rng.normal(loc=70, scale=12, size=500)

plt.figure(figsize=(8, 5))
plt.hist(values, bins=24, color="#8e7dff", edgecolor="white")
plt.title("A simple distribution")
plt.xlabel("Value")
plt.ylabel("Count")
plt.show()

Note ylim(0, 1) is just because accuracy is inherently between 0 and 1. If the ranges of different charts are not the same, one should not arbitrarily cut the y-axis to make one model look impressive; this is something to be mindful of when making reports.

5. Putting several graphs in one figure

Sometimes we don't want to open many images, we can use subplots. The example below puts a line graph and a histogram in the same figure. When I first used this, I also had a bit of trouble distinguishing fig And axcan be simply understood as fig is the entire sheet of paper,ax is each small area on the paper.

fig, ax = plt.subplots(1, 2, figsize=(12, 4))

ax[0].plot(x, y, color="#0071e3")
ax[0].set_title("sin(x)")
ax[0].set_xlabel("x")

ax[1].hist(values, bins=22, color="#34c759", edgecolor="white")
ax[1].set_title("Value distribution")
ax[1].set_xlabel("Value")

plt.tight_layout()
plt.show()

tight_layout() This thing is very convenient; it will automatically adjust the spacing for us, otherwise the labels sometimes collide. It's not a very deep function, but it's very practical.

6. Seaborn: A layer on top of Matplotlib

Seaborn is actually built on top of Matplotlib, but its default style looks a bit nicer and it works very well with Pandas DataFrame. For example, with the small DataFrame below, we can directly hand it over to Seaborn.

scores = pd.DataFrame({
    "study_hours": [2, 3, 4, 4, 5, 6, 7, 8, 9, 10],
    "score": [52, 55, 63, 60, 68, 74, 73, 82, 88, 91],
    "group": ["A", "A", "B", "A", "B", "B", "A", "B", "A", "B"],
})

sns.set_theme(style="whitegrid")
plt.figure(figsize=(8, 5))
sns.scatterplot(data=scores, x="study_hours", y="score", hue="group", s=90)
plt.title("Scores in two groups")
plt.show()

Here hue="group" will automatically change different groups into different colors. Seaborn has this feeling: for many common data visualizations, it has already set up some configurations for us. Of course, this doesn't mean Matplotlib is useless; Seaborn ultimately returns to Matplotlib, so using both together is more comfortable.

Another example that I think will be used easily in the future is the heatmap. For instance, let's first look at the correlation between several variables:

data = pd.DataFrame({
    "hours": [2, 3, 4, 5, 6, 7, 8, 9],
    "practice": [1, 2, 2, 3, 4, 5, 6, 7],
    "score": [50, 55, 59, 67, 72, 78, 84, 90],
})

plt.figure(figsize=(6, 4))
sns.heatmap(data.corr(numeric_only=True), annot=True, cmap="coolwarm", vmin=-1, vmax=1)
plt.title("Correlation heatmap")
plt.show()

annot=True will write the numbers directly in the boxes,cmap is the color. Red-blue color maps are common for visualizing correlation, but it should be reiterated that correlation is just a starting point, not the final answer.

7. For ML, the most worthwhile thing to plot

I think the most worthwhile thing to plot right at the beginning of training a model is the training loss and validation loss. Because this chart can roughly tell us whether the model is still learning or if it has started to overfit. The data below is just simulated; during actual training, just append the loss of each epoch.

epochs = np.arange(1, 11)
train_loss = [0.92, 0.75, 0.61, 0.49, 0.42, 0.36, 0.31, 0.28, 0.25, 0.22]
val_loss = [0.95, 0.79, 0.66, 0.56, 0.51, 0.49, 0.50, 0.54, 0.60, 0.69]

plt.figure(figsize=(9, 5))
plt.plot(epochs, train_loss, marker="o", label="training loss")
plt.plot(epochs, val_loss, marker="o", label="validation loss")
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.title("Training and validation loss")
plt.legend()
plt.grid(alpha=0.25)
plt.show()

If the training loss keeps decreasing, but the validation loss starts to rise later, that could be a signal of overfitting. At this point, there's no need to panic immediately; you can try early stopping, dropout, data augmentation, or simply check the data first.

Many ML problems are not issues with the model itself; after plotting the graph, you may find that the data issues are even bigger, haha.

So that's it for this episode. Matplotlib and Seaborn really have a lot more to offer, such as boxplot, pairplot, confusion matrix, 3D plot, animation, etc. I won't cover everything here, otherwise this episode would turn into a super long documentation.

Later, when we do model evaluation or Deep Learning, we will continue to use these charts, and we will discuss them again then.

In the next issue, we will continue with the Model Building section and gradually move towards Deep Learning and Computer Vision.

Everyone can first take a CSV file and try to draw a line graph, scatter plot, and heatmap. It's okay if you mess it up; running it a few more times will help you get familiar with it. See you all next time～

这一期，我们按既定计划介绍主要是Matplotlib这个画图的 library，和它的好朋友 Seaborn。在编纂这一期时，我改进了整个网站的post的layout系统，希望你们喜欢。我在这期也会变一些风格。

前面我们已经看了 Numpy 和 Pandas，基本上数据已经可以被我们拿到和处理了。那接下来最直接的问题就是：这些数据到底长什么样？这个时候如果只看一大堆 numbers，真的会有一点头晕，所以我们就需要画图。

其实 Matplotlib 对 ML 来说还是很重要的。画图不只是为了好看，很多时候 model 有没有过拟合，data 有没有奇怪的 distribution，或者两个 variables 到底有没有关系，一眼看图就会比看表格清楚很多。

当然，Matplotlib 可以画的东西非常多，这一期我们先把最常用的东西讲一下，之后自己写 project 的时候再慢慢补齐。

1. 先准备一下 Matplotlib

如果你的 environment 里面还没有 matplotlib，可以先在 terminal 或者 Colab 里面安装：

pip install matplotlib seaborn

然后我们一般这样 import。这里 pyplot 基本上就是我们最常用的画图 tool，大家都会把它叫成 plt，所以后面看到 plt 不用紧张。

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

2. 最基础的 line graph

先从最简单的 line graph 开始。假设我们有一条 sin curve。这里的 linspace 会帮我们在 -10 到 10 中间取 200 个 numbers，然后把它放进 np.sin() 里面就可以了。

x = np.linspace(-10, 10, 200)
y = np.sin(x)

plt.figure(figsize=(9, 5))
plt.plot(x, y, color="royalblue", linewidth=2, label="sin(x)")
plt.axhline(0, color="gray", linewidth=0.8)
plt.title("A simple sine curve")
plt.xlabel("x")
plt.ylabel("sin(x)")
plt.grid(alpha=0.25)
plt.legend()
plt.show()

运行之后你就会看到一个很标准的曲线。figure 是整个画布，plot 是画线，xlabel 和 ylabel 是坐标轴的名字，最后的 show() 就是把图真正显示出来。这里其实每一行都不是必须的，但是 title，label 和 grid 加上以后会清楚很多。自己看得懂的图，别人也更容易看懂。

3. Scatter plot：看两个东西有没有一点关系

ML 里面我觉得 scatter plot 很常见。例如我们想看看 study hours 和 score 有没有一点关系。为了让例子可以直接 run，这里用 random data 来模拟一下。

rng = np.random.default_rng(7)
study_hours = rng.uniform(1, 10, 90)
score = 42 + study_hours * 5.4 + rng.normal(0, 7, 90)

plt.figure(figsize=(8, 5))
plt.scatter(study_hours, score, color="#ff7a59", alpha=0.72)
plt.title("Study hours and score")
plt.xlabel("Study hours")
plt.ylabel("Score")
plt.grid(alpha=0.2)
plt.show()

这个图里面每一个 dot 就是一条 data。alpha 是透明度，如果 dots 很多的话，加一点透明度会比较容易看出哪里特别密集。

这个例子大概会有一个向上的 trend，当然这不代表 study longer 就一定 score higher，只是它们在这组 data 里面看起来有一个 correlation。这个东西要分清楚，correlation 不一定是 causation，后面做 data analysis 的时候还是要小心。

4. Bar chart 和 histogram

Bar chart 很适合比较不同 categories，例如不同 models 的 accuracy。Histogram 则是看一个 number 的 distribution。这个两个图看起来很基础，但是 model evaluation 的时候真的会用到。

models = ["Logistic Regression", "Decision Tree", "Random Forest"]
accuracy = [0.79, 0.83, 0.89]

plt.figure(figsize=(9, 5))
plt.bar(models, accuracy, color=["#90caf9", "#81c784", "#ffcc80"])
plt.ylim(0, 1)
plt.title("Model accuracy comparison")
plt.ylabel("Accuracy")
plt.show()

values = rng.normal(loc=70, scale=12, size=500)

plt.figure(figsize=(8, 5))
plt.hist(values, bins=24, color="#8e7dff", edgecolor="white")
plt.title("A simple distribution")
plt.xlabel("Value")
plt.ylabel("Count")
plt.show()

注意 ylim(0, 1) 只是因为 accuracy 本来就是 0 到 1 之间。如果不同 charts 的 range 不一样，不能为了让一个 model 看起来厉害就随便把 y axis 截得很奇怪，这个在做 report 的时候要稍微注意一下。

5. 一张图里面放几个 graph

有时候我们不想开很多图片，可以用 subplots。下面这个例子把 line graph 和 histogram 放在同一张 figure 里面。我刚开始用这个的时候也有点分不清 fig 和 ax，可以先简单理解成 fig 是整张纸，ax 是纸上面每一个小区域。

fig, ax = plt.subplots(1, 2, figsize=(12, 4))

ax[0].plot(x, y, color="#0071e3")
ax[0].set_title("sin(x)")
ax[0].set_xlabel("x")

ax[1].hist(values, bins=22, color="#34c759", edgecolor="white")
ax[1].set_title("Value distribution")
ax[1].set_xlabel("Value")

plt.tight_layout()
plt.show()

tight_layout() 这个东西很方便，它会自动帮我们挪一下 spacing，不然 label 有时候会撞在一起。不是一个很深的 function，但是很实用。

6. Seaborn：在 Matplotlib 上面再包一层

Seaborn 其实还是建立在 Matplotlib 上面，只是它默认的 style 会好看一点，而且和 Pandas DataFrame 配合得很方便。比如下面这个小 DataFrame，我们可以直接交给 Seaborn。

scores = pd.DataFrame({
    "study_hours": [2, 3, 4, 4, 5, 6, 7, 8, 9, 10],
    "score": [52, 55, 63, 60, 68, 74, 73, 82, 88, 91],
    "group": ["A", "A", "B", "A", "B", "B", "A", "B", "A", "B"],
})

sns.set_theme(style="whitegrid")
plt.figure(figsize=(8, 5))
sns.scatterplot(data=scores, x="study_hours", y="score", hue="group", s=90)
plt.title("Scores in two groups")
plt.show()

这里 hue="group" 会把不同 group 自动变成不同 color。Seaborn 就是这种感觉：很多常见的 data visualization，它已经把一些设置帮我们做好了。当然不是说 Matplotlib 就没用了，Seaborn 最后还是会回到 Matplotlib，所以两个一起会比较舒服。

再举一个我觉得以后很容易用到的 heatmap。比如我们先看几个 variables 之间的 correlation：

data = pd.DataFrame({
    "hours": [2, 3, 4, 5, 6, 7, 8, 9],
    "practice": [1, 2, 2, 3, 4, 5, 6, 7],
    "score": [50, 55, 59, 67, 72, 78, 84, 90],
})

plt.figure(figsize=(6, 4))
sns.heatmap(data.corr(numeric_only=True), annot=True, cmap="coolwarm", vmin=-1, vmax=1)
plt.title("Correlation heatmap")
plt.show()

annot=True 会把 number 直接写在格子上，cmap 就是颜色。红蓝这种 color map 看 correlation 很常见，但是这里也要再说一次，相关性只是一个开始，不是最后的答案。

7. 对 ML 来说，最值得画的一个东西

我觉得刚开始 train model 的时候，最值得马上画出来的就是 training loss 和 validation loss。因为这张图可以大概告诉我们 model 有没有还在 learning，还是已经有一点 overfitting。下面只是模拟的数据，真正 training 的时候把每一个 epoch 的 loss append 进去就好了。

epochs = np.arange(1, 11)
train_loss = [0.92, 0.75, 0.61, 0.49, 0.42, 0.36, 0.31, 0.28, 0.25, 0.22]
val_loss = [0.95, 0.79, 0.66, 0.56, 0.51, 0.49, 0.50, 0.54, 0.60, 0.69]

plt.figure(figsize=(9, 5))
plt.plot(epochs, train_loss, marker="o", label="training loss")
plt.plot(epochs, val_loss, marker="o", label="validation loss")
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.title("Training and validation loss")
plt.legend()
plt.grid(alpha=0.25)
plt.show()

如果 training loss 一直下降，但是 validation loss 后面开始往上跑，那就可能是 overfitting 的一个 signal。这个时候不一定要马上 panic，可以试一下 early stopping，dropout，data augmentation 或者干脆先检查一下 data。

很多 ML 问题不是 model 本身的问题，图画出来以后才会发现原来 data 的问题更大，哈哈。

那么这一期就先到这里。Matplotlib 和 Seaborn 真的还有很多东西，比如 boxplot，pairplot，confusion matrix，3D plot，animation 等等，这里就先不全部讲完了，不然这期会变成一个超级长的 documentation。

之后我们做 model evaluation 或者 Deep Learning 的时候还会继续用到这些 charts，到时候再拿出来讲。

下一期我们会继续走 Model Building 的部分，也会慢慢往 Deep Learning 和 Computer Vision 靠近。

大家可以先自己拿一个 csv，试着画出 line graph，scatter plot 和 heatmap，画坏了也没关系，多 run 几次就会熟悉了。大家下期见～

Matplotlib, Seaborn and Plotting ToolsMatplotlib, Seaborn and Plotting Tools