获取apache日志作为csv文件

axkjgtzd  于 11个月前  发布在  Apache
关注(0)|答案(3)|浏览(125)

有没有办法保存所有的Apache日志作为CSV文件?

access.log->access_log.csv
error.log->error_log.csv

字符串

vcirk6k6

vcirk6k61#

您可以定义自定义日志格式,将Apache日志直接转换为逗号分隔格式。
您可能需要花一段时间来处理这个问题,以找到正确的方法。例如,您可能希望使用"'作为字段分隔符,以防止字段值中的逗号破坏CSV。

2g32fytz

2g32fytz2#

如果你有问题,你想看看过去写的日志文件,或日志文件从apache服务器,你没有访问配置文件,或者如果你出于其他原因不想改变日志文件格式:
我已经写了一个little linux shell sed script,它将默认的apache日志文件转换为libre office calc可以读取的格式:

#!/bin/bash

#reformat apache's access logs, so that they can be interpreted as csv files, 
# with space as column delimiter and double quotes to bind together things
# that contain spaces but represent single columns.

# 1)  add a doublequote at the begining of the line. first column is the ip adress. 
#     ip-adresses that have 3 digits in every group but the first could be interpreted as numbers 
#     with the dots marking groups of thousands.

# 2a) end the ip-adress with quotes
# 2b) surround the second (to me unknown) column thats always just "-" and the
#     third column which is the username with quotes
# 2c) reformat the date from "[09/Jul/2012:11:17:47" to "09.Jul 2012 11:17:47"

# 3)  remove the string "+0200]" (replace it with doublequotes to end the date column)

# 4)  the string that contains the command (5th column) sometimes contains string representation 
#     of binary rubish. thats no problem as long as this does not contain a doublequote which 
#     will mess up the column zoning. According to my web searches, csv columns should allow to 
#     contain doublequotes if they are escaped with a backslash. Although this is the case with
#     these problematic strings, Libre Office does not accept it that way. therefore we escape every 
#     doublequote with a doubleqoute, which is the other valid option according to csv specifications,
#     and libre office does accept that one. More technical: we replace every doublequote that does
#     neither have a space or another doublequote before it, neither after it, with two doublequotes.

sed \
-e 's/^/"/' \
-e 's/ \([^ ]\{1,\}\) \([^ ]\{1,\}\) \[\([0-9]\{1,2\}\)\/\([a-zA-Z]\{1,3\}\)\/\([0-9]\{1,4\}\):/" "\1" "\2" "\3.\4 \5 /' \
-e 's/ +0200\] /" /' \
-e 's/\([^" ]\)"\([^" ]\)/\1""\2/g'

字符串

q0qdq0h2

q0qdq0h23#

这实际上只是@kaefert的答案的一个修改。我相信有一个更干净的方法来做到这一点,但这工作得很好。

alias aplogcsv="sed -e 's/^/\"/' \
                -e 's/:\([0-9]\{1,3\}\.\)\([0-9]\{1,3\}\.\)\([0-9]\{1,3\}\.\)\([0-9]\{1,3\}\)/\",\"\1\2\3\4/' \
                -e 's/ \([^ ]\{1,\}\) \([^ ]\{1,\}\) \[\([0-9]\{1,2\}\)\/\([a-zA-Z]\{1,3\}\)\/\([0-9]\{1,4\}\):/\",\"\1\" \"\2\" \"\3 \4 \5\",\" /' \
                -e 's/ \([0-9]\{1,2\}\):\([0-9]\{1,2\}\):\([0-9]\{1,2\}\)/\1:\2:\3/' \
                -e 's/ $(date +%z)\] /\",/' \
                -e 's/\"GET /\"GET\",\"/g' \
                -e 's/\"POST /\"POST\",\"/g' \
                -e 's/ HTTP\/1.1\" \([0-9]\{1,3\}\) \([0-9]\{1,4\}\) /\",\"HTTP\/1.1\",\1,\2,/' \
                -e 's/\"-\" //g'"

字符串
然后我这样使用它:

aplogcsv access.log > ~/access.log.csv


但它也很容易使用,就像这样:

grep "25/Jan/2019" access.log | aplogcsv > ~/20190125.access.log.csv

相关问题