CLANA 是一个分类器分析工具包。
项目描述
克拉纳
clana
是一个库和命令行应用程序,用于可视化具有大量类的分类器的混淆矩阵。clana 的两个关键贡献是混淆矩阵排序 (CMO),如卷积神经网络架构的分析和优化第 5 章所述,以及实现它的优化算法。CMO 技术可以应用于任何多类分类器,并有助于了解哪些类组最相似。
安装
推荐的安装clana的方法是:
$ pip install clana --user --upgrade
如果你想要最新版本:
$ git clone https://github.com/MartinThoma/clana.git; cd clana
$ pip install -e . --user
用法
$ clana --help
Usage: clana [OPTIONS] COMMAND [ARGS]...
Clana is a toolkit for classifier analysis.
See https://arxiv.org/abs/1707.09725, Chapter 4.
Options:
--version Show the version and exit.
--help Show this message and exit.
Commands:
distribution Get the distribution of classes in a dataset.
get-cm Generate a confusion matrix from predictions and ground...
get-cm-simple Generate a confusion matrix.
visualize Optimize and visualize a confusion matrix.
可视化命令为您提供如下图像:
MNIST 示例
$ cd docs/
$ python mnist_example.py # creates `train-pred.csv` and `test-pred.csv`
$ clana get-cm --gt gt-train.csv --predictions train-pred.csv --n 10
2019-09-14 09:47:30,655 - root - INFO - cm was written to 'cm.json'
$ clana visualize --cm cm.json --zero_diagonal
Score: 13475
2019-09-14 09:49:41,593 - root - INFO - n=10
2019-09-14 09:49:41,593 - root - INFO - ## Starting Score: 13475.00
2019-09-14 09:49:41,594 - root - INFO - Current: 13060.00 (best: 13060.00, hot_prob_thresh=100.0000%, step=0, swap=False)
[...]
2019-09-14 09:49:41,606 - root - INFO - Current: 9339.00 (best: 9339.00, hot_prob_thresh=100.0000%, step=238, swap=False)
Score: 9339
Perm: [0, 6, 5, 8, 3, 2, 1, 7, 9, 4]
2019-09-14 09:49:41,639 - root - INFO - Classes: [0, 6, 5, 8, 3, 2, 1, 7, 9, 4]
Accuracy: 93.99%
2019-09-14 09:49:41,725 - root - INFO - Save figure at '/home/moose/confusion_matrix.tmp.pdf'
2019-09-14 09:49:41,876 - root - INFO - Found threshold for local connection: 398
2019-09-14 09:49:41,876 - root - INFO - Found 9 clusters
2019-09-14 09:49:41,877 - root - INFO - silhouette_score=-0.012313948323292875
1: [0]
1: [6]
1: [5]
1: [8]
1: [3]
1: [2]
1: [1]
2: [7, 9]
1: [4]
这给
标签操作
准备一个labels.csv
必须有标题行的:
$ clana visualize --cm cm.json --zero_diagonal --labels mnist/labels.csv
数据分布
$ clana distribution --gt gt.csv --labels labels.csv [--out out/] [--long]
每个标签打印一行,例如
60% cat (56789 elements)
20% dog (12345 elements)
5% mouse (1337 elements)
1% tux (314 elements)
如果--out
指定,则创建水平条形图。第一个栏是最常见的类,第二个栏是第二个最常见的类,...
它使用短标签,除了--long
添加到命令中。
可视化
查看可视化
用作图书馆
>>> import numpy as np
>>> arr = np.array([[9, 4, 7, 3, 8, 5, 2, 8, 7, 6],
[4, 9, 2, 8, 5, 8, 7, 3, 6, 7],
[7, 2, 9, 1, 6, 3, 0, 8, 5, 4],
[3, 8, 1, 9, 4, 7, 8, 2, 5, 6],
[8, 5, 6, 4, 9, 6, 3, 7, 8, 7],
[5, 8, 3, 7, 6, 9, 6, 4, 7, 8],
[2, 7, 0, 8, 3, 6, 9, 1, 4, 5],
[8, 3, 8, 2, 7, 4, 1, 9, 6, 5],
[7, 6, 5, 5, 8, 7, 4, 6, 9, 8],
[6, 7, 4, 6, 7, 8, 5, 5, 8, 9]])
>>> from clana.optimize import simulated_annealing
>>> result = simulated_annealing(arr)
>>> result.cm
array([[9, 8, 7, 6, 5, 4, 3, 2, 1, 0],
[8, 9, 8, 7, 6, 5, 4, 3, 2, 1],
[7, 8, 9, 8, 7, 6, 5, 4, 3, 2],
[6, 7, 8, 9, 8, 7, 6, 5, 4, 3],
[5, 6, 7, 8, 9, 8, 7, 6, 5, 4],
[4, 5, 6, 7, 8, 9, 8, 7, 6, 5],
[3, 4, 5, 6, 7, 8, 9, 8, 7, 6],
[2, 3, 4, 5, 6, 7, 8, 9, 8, 7],
[1, 2, 3, 4, 5, 6, 7, 8, 9, 8],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]])
>>> result.perm
array([2, 7, 0, 4, 8, 9, 5, 1, 3, 6])
您可以可视化result.cm
并使用result.perm
以相同的顺序获取标签:
# Just some example labels
# ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10']
>>> labels = [str(el) for el in range(11)]
>>> np.array(labels)[result.perm]
array(['2', '7', '0', '4', '8', '9', '5', '1', '3', '6'], dtype='<U2')
项目详情
下载文件
下载适用于您平台的文件。如果您不确定要选择哪个,请了解有关安装包的更多信息。
源分布
clana-0.4.1.tar.gz
(21.5 kB
查看哈希)
内置分布
clana-0.4.1-py3-none-any.whl
(23.9 kB
查看哈希)