将时间序列CSV转换为Prometheus的OpenMetrics

i86rm4rw  于 2023-04-03  发布在  其他
关注(0)|答案(1)|浏览(146)

我有一个时间序列数据集,格式如下:

time,id_station,s0,s1,s2,s3
2019-03-01T00:00:00,st0,0.093684,0.601416,0.020954,0.179308
2019-03-01T00:00:10,st0,0.181069,0.323754,0.624349,0.043567
...

我需要将此csv文件转换为OpenMetrics格式,以便能够将其加载到Prometheus。我尝试了GitHub上的几个工具,如csv-exporter,但它不适合我。
如何将此csv数据集转换为OpenMetrics格式?

owfi6suc

owfi6suc1#

您可以自己转换指标。
下面是一个Python函数的例子:它将拆分格式csv字符串并基于它生成度量。

from typing import List
import datetime

METRIC_COMMON_PREFIX = 'my_sensor_data_s'

def get_metrics(csvline: str) -> List[str]:
    result = []
    values = csvline.split(',')
    my_timestamp = int(datetime.datetime.fromisoformat(values[0]).timestamp())

    for i in range(2, len(values)):
        result.append(f'{METRIC_COMMON_PREFIX}{i - 1}{{id_station="{values[1]}"}} {values[i]} {my_timestamp}')

    return result

print('\n'.join(get_metrics("2019-03-01T00:00:00,st0,0.093684,0.601416,0.020954,0.179308")))
print('\n'.join(get_metrics('2019-03-02T00:00:10,st0,0.181069,0.323754,0.624349,0.043567')))

输出示例:

my_sensor_data_s1{id_station="st0"} 0.093684 1551391200
my_sensor_data_s2{id_station="st0"} 0.601416 1551391200
my_sensor_data_s3{id_station="st0"} 0.020954 1551391200
my_sensor_data_s4{id_station="st0"} 0.179308 1551391200
my_sensor_data_s1{id_station="st0"} 0.181069 1551477610
my_sensor_data_s2{id_station="st0"} 0.323754 1551477610
my_sensor_data_s3{id_station="st0"} 0.624349 1551477610
my_sensor_data_s4{id_station="st0"} 0.043567 1551477610

这里假设秒之后的列是不同传感器的值,因此为每列生成不同的度量。
要转换整个文件,您可以使用这样的脚本:

with open('in.csv') as infile:
    with open('out.prom', 'w') as outfile:
        for line in infile:
            metrics = get_metrics(line.rstrip())
            for metric in metrics:
                outfile.write(metric + '\n')

相关问题