elasticsearch 从私有VPC中的OpenSearch导出数据并将其导入到本地运行容器- aws opensearch

5uzkadbs  于 2022-12-11  发布在  ElasticSearch
关注(0)|答案(1)|浏览(284)

我在一个私人的vpc中使用aws OpenSearch。我在某个索引下有大约10000个条目。
对于本地开发,我正在运行一个本地OpeanSearch容器,我想将OpenSearch服务中的所有条目导出到我的本地容器中。
我可以从OpeanSerch API获取所有条目,但是响应的格式与执行_bulk操作时的格式不同。
有人能告诉我我该怎么做吗?

2w3rbyxf

2w3rbyxf1#

Anna ,
There are different strategies you can take to accomplish this, considering the fact that your domain is running in a private VPC.

Option 1: Exporting and Importing Snapshots

From the security standpoint, this is the recommended option, as you are moving entire indices out of the service without exposing the data. Please follow the AWS official documentation about how to create custom index snapshots . Once you complete the steps, you will have an index snapshot stored on an Amazon S3 bucket. After this, you can securely download the index snapshot to your local machine, then follow the instructions on the official OpenSearch documentation about how to restore the index snapshots .

Option 2: Using VPC Endpoints

Another way for you to export the data from your OpenSearch domain is accessing the data via a alternate endpoint using the VPC Endpoints feature from AWS OpenSearch. It allows you to to expose additional endpoints running on public or private subnets within the same VPC, different VPC, or different AWS accounts. In this case, you are essentially create a venue to access the OpenSearch REST APIs outside of the private VPC, to which you need to take care of who other than you will be able to do so as well. Please follow the best practices related to secure endpoints if you follow this option.

Option 3: Using the ElasticDump Open Source Utility

The ElasticDump utility allows you to retrieve data from Elasticsearch/OpenSearch clusters in a format of your preference, and then import that data back to another cluster. It is a very flexible way for you to move data around—but it requires the utility to access the REST API endpoints from the cluster. Run this utility in a bastion server that has ingress access to your OpenSearch domain in the private VPC. Keep in mind, though, that AWS doesn't provide any support to this utility, and you must use it at your own risk.
I hope that helps with your question. Let us know if you need any more help on this. 🙂

相关问题