elasticsearch 获取基本Eland示例的错误(从本地安装的ELK Docker容器加载索引)

2w3rbyxf  于 2022-12-11  发布在  ElasticSearch
关注(0)|答案(1)|浏览(311)

我们在基于this example的docker中安装了ELK。例如:

docker run -d --name elasticsearchdb --net es-stack-network -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:6.8.13

docker run -d --name kibana-es-ui --net es-stack-network -e "ELASTICSEARCH_URL=http://elasticsearchdb:9200" -p 5601:5601 kibana:6.8.13

然后,我们使用基本的内置数据集设置Elastic,包括默认提供的航班数据集。
然后,我们尝试使用Eland将数据拉入 Dataframe ,我认为我们正确地遵循了文档。
而是用代码:

import eland as ed
    index_name = 'flights'
    ed_df = ed.DataFrame('localhost:9200', index_name)

我们得到这个错误:

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\elastic_transport\client_utils.py:198, in url_to_node_config(url)
    192     raise ValueError(f"Could not parse URL {url!r}") from None
    194 if any(
    195     component in (None, "")
    196     for component in (parsed_url.scheme, parsed_url.host, parsed_url.port)
    197 ):
--> 198     raise ValueError(
    199         "URL must include a 'scheme', 'host', and 'port' component (ie 'https://localhost:9200')"
    200     )
    202 headers = {}
    203 if parsed_url.auth:

ValueError: URL must include a 'scheme', 'host', and 'port' component (ie 'https://localhost:9200')

所以当我们加上http://时,就像这样:

import eland as ed
    index_name = 'flights'
    ed_df = ed.DataFrame('http://localhost:9200', index_name)

我们得到这个错误:

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\elastic_transport\_node\_http_urllib3.py:199, in Urllib3HttpNode.perform_request(self, method, target, body, headers, request_timeout)
    191         err = ConnectionError(str(e), errors=(e,))
    192     self._log_request(
    193         method=method,
    194         target=target,
   (...)
    197         exception=err,
    198     )
--> 199     raise err from None
    201 meta = ApiResponseMeta(
    202     node=self.config,
    203     duration=duration,
   (...)
    206     headers=response_headers,
    207 )
    208 self._log_request(
    209     method=method,
    210     target=target,
   (...)
    214     response=data,
    215 )

ConnectionError: Connection error caused by: ConnectionError(Connection error caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))))

所以我想,好吧,也许它是默认的HTTPS服务的一些原因,也许不相关,但在日志中我看到:

05T17:17:04.734Z", "log.level": "WARN", "message":"received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/172.18.0.3:9200, remoteAddress=/172.18.0.1:59788}", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[21683dc12cff][transport_worker][T#14]","log.logger":"org.elasticsearch.xpack.security.transport.netty4.SecurityNetty4HttpServerTransport","elasticsearch.cluster.uuid":"XuzqXMk_QgShA3L5HnfXgw","elasticsearch.node.id":"H1CsKboeTyaFFjk2-1nw2w","elasticsearch.node.name":"21683dc12cff","elasticsearch.cluster.name":"docker-cluster"}

因此我尝试将http替换为https,并得到以下错误:

TlsError: TLS error caused by: TlsError(TLS error caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)))

因此,我查找此错误,并找到this thread,其中显示执行以下操作:

import ssl
from elasticsearch.connection import create_ssl_context

ssl_context = create_ssl_context(<use `cafile`, or `cadata` or `capath` to set your CA or CAs)
context.check_hostname = False
context.verify_mode = ssl.CERT_NONE

es = Elasticsearch('localhost', ssl_context=context, timeout=60

但这没有帮助,因为Eland在内部处理elasticsearch示例化,我不控制它。
这是一个非常基本的场景,所以我相信解决方案一定比这一切简单得多。我该怎么做才能使它工作呢?

cs7cruho

cs7cruho1#

对于那些还在苦苦挣扎的人来说,下面的代码对我来说是有用的,它使用了Docker/docker-compose的本地弹性集群:
Following this guide您可以使用以下命令在本地创建http_ca. crt文件:

docker cp es01:/usr/share/elasticsearch/config/certs/http_ca.crt .

您可以在创建es_client时使用http_ca. crt文件:

from elasticsearch import Elasticsearch
es_client = Elasticsearch("https://localhost:9200",
                          ca_certs="/path/to/http_ca.crt",
                          basic_auth=("[elastic username]",
                                      "[elastic password]"))

然后使用es_client连接eland:

import eland as ed
df = ed.DataFrame(es_client=es_client, es_index_pattern="[Your index]")
df.head()

相关问题