Class Order shuffling Results in Fig. 6

[Ze-Yang]

Hi, configs/ade20k/panoptic-segmentation contains 9 random class orders + 1 default class order. May I know whether the results in Fig. 6 averaged over 10 random class orders are obtained using these ten config files?

I notice that the default order yields over 40 on base classes, while the top base result in Fig. 6 is lower than 40. Could you clarify on this issue? Thanks.

Could you kindly share the results for drawing the boxplot?

[qjadud1994]

Hi.

Hi, configs/ade20k/panoptic-segmentation contains 9 random class orders + 1 default class order. May I know whether the results in Fig. 6 averaged over 10 random class orders are obtained using these ten config files?

yes, I used the 9 random class orderings using the config files.

I notice that the default order yields over 40 on base classes, while the top base result in Fig. 6 is lower than 40. Could you clarify on this issue? Thanks. Could you kindly share the results for drawing the boxplot?

The below is our drawing code.

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns

plt.style.use('default')
plt.rcParams['figure.figsize'] = (16, 12)
plt.rcParams['font.size'] = 18

data = {
    "ECLIPSE": {
        "base":  [41.4, 34.6, 32.6, 34.3, 35.0, 34.5, 32.8, 32.3, 32.5, 33.2],
        "novel": [18.8, 23.7, 21.1, 22.8, 23.8, 22.8, 21.5, 21.2, 22.4, 24.4],
        "all":   [33.9, 31.0, 28.7, 30.5, 31.3, 30.6, 29.1, 28.6, 29.1, 30.3],
        },

    "CoMFormer": {
        "base":  [36.0, 31.7, 30.7, 32.4, 32.5, 32.4, 29.6, 30.1, 31.6, 30.9],
        "novel": [17.1, 18.5, 17.1, 16.7, 16.7, 17.9, 15.1, 17.3, 16.4, 18.5],
        "all":   [29.7, 27.3, 26.2, 27.2, 27.2, 27.6, 24.7, 25.9, 26.5, 26.7],
        },

    "PLOP": {
        "base":  [30.5, 31.4, 30.4, 31.9, 31.7, 32.3, 29.2, 30.9, 31.2, 30.5],
        "novel": [17.5, 18.5, 14.7, 16.2, 15.5, 16.1, 14.7, 16.6, 15.5, 17.0],
        "all":   [26.1, 27.0, 24.5, 26.3, 26.0, 26.9, 23.7, 25.5, 25.3, 25.0],
        },

    "MiB": {
        "base":  [27.1, 28.8, 27.7, 28.6, 28.4, 28.8, 27.3, 27.3, 27.2, 28.6],
        "novel": [10.0, 14.4, 14.1, 11.8, 12.3, 11.3, 12.7, 14.6, 11.4, 12.9],
        "all":   [21.4, 21.7, 20.8, 20.7, 20.7, 20.6, 20.1, 20.7, 19.6, 21.0],
        },
}

data_str = []

for method, result in data.items():
    for type, PQ_list in result.items():
        for PQ in PQ_list:
            data_str.append(f"{method},{type},{PQ}")

str_to_list = [x.split(',') for x in data_str]
str_to_list = [[p1, p2, float(p3)] for p1, p2, p3 in str_to_list]

df = pd.DataFrame(str_to_list)
df.columns = ["method", "type", "PQ"]
print(df.head(10))

plt.figure()
sns.boxplot(
    y='PQ', 
    x='type', 
    data=df, 
    palette="colorblind",
    hue='method',
    gap=0.1,
    showfliers=False,
    linewidth=2,
    width=0.75,
    )

plt.grid(True, axis='y', color='gray', alpha=0.2)
plt.axvline(0.5, 0.0, 100.0, color='gray', alpha=0.2)
plt.axvline(1.5, 0.0, 100.0, color='gray', alpha=0.2)
plt.legend(title='')
plt.xlabel("")
plt.ylabel("PQ (%)")
plt.show()