如何将camel google pubsub组件连接到pubsub模拟器？

to94eoyn  于 2021-07-06  发布在  Java
关注(0)|答案(2)|浏览(531)

我已经尝试了几个小时，但都没有成功地获取我的spring boot应用程序的camel路由（通过camel组件camel-google-pubsub连接到google-pubsub模拟器的本地示例）。
我的模拟器正在运行：

❯ gcloud beta emulators pubsub start --host-port=localhost:8085 --project=local
Executing: /usr/lib/google-cloud-sdk/platform/pubsub-emulator/bin/cloud-pubsub-emulator --host=localhost --port=8085
[pubsub] This is the Google Pub/Sub fake.
[pubsub] Implementation may be incomplete or differ from the real system.
[pubsub] déc. 01, 2020 6:14:32 PM com.google.cloud.pubsub.testing.v1.Main main
[pubsub] INFO: IAM integration is disabled. IAM policy methods and ACL checks are not supported
[pubsub] déc. 01, 2020 6:14:33 PM com.google.cloud.pubsub.testing.v1.Main main
[pubsub] INFO: Server started, listening on 8085

我的环境变量正常： PUBSUB_EMULATOR_HOST=localhost:8085 我已经残疾了 GOOGLE_APPLICATION_CREDENTIALS 为了避免连接到真实的pubsub示例
我的 Camel 路线：

@Component
public class MapperInRoute extends ESBRouteBuilder {

    private final String dlFuncTopic;
    private final String subscription;
    private final String routeName;
    private final String getLinksTopic;

    public MapperInRoute(
        final MapperInRouteConfiguration config,
        @Value("${routes.get-links-topic}") final String getLinksTopic
    ) {
        this.dlFuncTopic = config.getDlFuncTopic();
        this.subscription = config.getSubscription();
        this.routeName = config.getName();
        this.getLinksTopic = getLinksTopic;
    }

    @Override
    public void configure() {
        from(format("google-pubsub:{{gcp.pubsub.project-id}}:%s", subscription))
            .routeId(routeName)
            // my route code
            .to(format("google-pubsub:{{gcp.pubsub.project-id}}:%s", getLinksTopic))
        ;
    }
}

我的 application-dev.yml :

gcp.pubsub.project-id: local

spring.cloud.gcp.pubsub:
    emulator-host: localhost:8085
    project-id: local

camel.component.google-pubsub.connection-factory.serviceURL: localhost:8085

最后，当我尝试在本地运行我的应用程序时，我的控制台输出如下：

2020-12-03 12:17:58 INFO   | Starting Application using Java 11.0.2 on lasouche-vm with PID 78394 (/home/mathieu/git/rocher/omnichannel/esb/poc-omega/target/classes started by mathieu in /home/mathieu/git/rocher/omnichannel/esb/poc-omega)
2020-12-03 12:17:58 INFO   | The following profiles are active: dev
2020-12-03 12:17:59 INFO   | Type converters loaded (core: 195, classpath: 10)
2020-12-03 12:17:59 WARN   | UT026010: Buffer pool was not set on WebSocketDeploymentInfo, the default pool will be used
2020-12-03 12:17:59 INFO   | Initializing Spring embedded WebApplicationContext
2020-12-03 12:17:59 INFO   | Root WebApplicationContext: initialization completed in 1696 ms
2020-12-03 12:18:00 INFO   | Registered '/actuator/jolokia' to jolokia-actuator-endpoint
2020-12-03 12:18:00 INFO   | Initialising hawtio services
2020-12-03 12:18:00 INFO   | Configuration will be discovered via system properties
2020-12-03 12:18:00 INFO   | Welcome to Hawtio 2.10.0
2020-12-03 12:18:00 INFO   | Initializing ExecutorService 'applicationTaskExecutor'
2020-12-03 12:18:00 INFO   | Adding welcome page: class path resource [static/index.html]
2020-12-03 12:18:00 INFO   | The default project ID is omega-lab-e5727d
2020-12-03 12:18:00 INFO   | Failed to detect whether we are running on Google Compute Engine.
2020-12-03 12:18:00 WARN   | No core credentials are set. Service-specific credentials (e.g., spring.cloud.gcp.pubsub.credentials.*) should be used if your app uses services that require credentials.
java.io.IOException: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
    at com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentials(DefaultCredentialsProvider.java:134)
    at com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:119)
    at com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:91)
    at com.google.api.gax.core.GoogleCredentialsProvider.getCredentials(GoogleCredentialsProvider.java:67)
    at org.springframework.cloud.gcp.core.DefaultCredentialsProvider.<init>(DefaultCredentialsProvider.java:110)
    at org.springframework.cloud.gcp.autoconfigure.core.GcpContextAutoConfiguration.googleCredentials(GcpContextAutoConfiguration.java:61)
    [...]
2020-12-03 12:18:04 INFO   | Initializing ExecutorService 'pubsubPublisherThreadPool'
2020-12-03 12:18:05 INFO   | Initializing ExecutorService 'pubsubSubscriberThreadPool'
2020-12-03 12:18:05 INFO   | Initializing ExecutorService 'pubSubAcknowledgementExecutor'
2020-12-03 12:18:05 INFO   | Exposing 7 endpoint(s) beneath base path '/actuator'
2020-12-03 12:18:06 INFO   | Initialized CamelHttpTransportServlet[name=CamelServlet, contextPath=]
2020-12-03 12:18:06 INFO   | Starting hawtio authentication filter, JAAS authentication disabled
2020-12-03 12:18:06 INFO   | starting server: Undertow - 2.2.2.Final
2020-12-03 12:18:06 INFO   | XNIO version 3.8.0.Final
2020-12-03 12:18:06 INFO   | XNIO NIO Implementation Version 3.8.0.Final
2020-12-03 12:18:06 INFO   | JBoss Threads version 3.1.0.Final
2020-12-03 12:18:06 INFO   | Undertow started on port(s) 8080 (http)
2020-12-03 12:18:06 INFO   | Loading additional Camel XML routes from: classpath:camel/*.xml
2020-12-03 12:18:06 INFO   | Loading additional Camel XML rests from: classpath:camel-rest/*.xml
2020-12-03 12:18:06 INFO   | Apache Camel 2.25.2 (CamelContext: camel-1) is starting
2020-12-03 12:18:06 INFO   | JMX is enabled
2020-12-03 12:18:06 INFO   | StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
2020-12-03 12:18:13 INFO   | Apache Camel 2.25.2 (CamelContext: camel-1) is shutting down
2020-12-03 12:18:13 INFO   | Apache Camel 2.25.2 (CamelContext: camel-1) uptime 7.045 seconds
2020-12-03 12:18:13 INFO   | Apache Camel 2.25.2 (CamelContext: camel-1) is shutdown in 0.080 seconds
2020-12-03 12:18:13 INFO   | stopping server: Undertow - 2.2.2.Final
2020-12-03 12:18:13 INFO   | Destroying hawtio authentication filter
2020-12-03 12:18:13 INFO   | Destroyed CamelHttpTransportServlet[CamelServlet]
2020-12-03 12:18:13 INFO   | Destroying hawtio services
2020-12-03 12:18:13 INFO   | 

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2020-12-03 12:18:13 ERROR  | Application run failed
org.apache.camel.RuntimeCamelException: org.apache.camel.FailedToCreateRouteException: Failed to create route esb-poc-omega-consumer: Route(esb-poc-omega-consumer)[[From[file:/data/loyalty/from/... because of Failed to create Producer for endpoint: google-pubsub://local:POC.OMEGA.CONSUMER.OUT. Reason: java.io.IOException: The Application Default Credentials are not available. They are available if running on Google App Engine, Google Compute Engine, or Google Cloud Shell. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
    at org.apache.camel.util.ObjectHelper.wrapRuntimeCamelException(ObjectHelper.java:1826)
    at org.apache.camel.spring.SpringCamelContext.start(SpringCamelContext.java:136)
    at org.apache.camel.spring.SpringCamelContext.onApplicationEvent(SpringCamelContext.java:174)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:203)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:196)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:161)
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:426)
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:383)
    at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:945)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:591)
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:144)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:767)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759)
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:426)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:326)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1309)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1298)
    at com.grouperocher.esb.pocfuse7.Application.main(Application.java:13)
Caused by: org.apache.camel.FailedToCreateRouteException: Failed to create route esb-poc-omega-consumer: Route(esb-poc-omega-consumer)[[From[file:/data/loyalty/from/... because of Failed to create Producer for endpoint: google-pubsub://local:POC.OMEGA.CONSUMER.OUT. Reason: java.io.IOException: The Application Default Credentials are not available. They are available if running on Google App Engine, Google Compute Engine, or Google Cloud Shell. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
    at org.apache.camel.impl.RouteService.warmUp(RouteService.java:147)
    at org.apache.camel.impl.DefaultCamelContext.doWarmUpRoutes(DefaultCamelContext.java:3954)
    at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:3861)
    at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRoutes(DefaultCamelContext.java:3647)
    at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:3488)
    at org.apache.camel.impl.DefaultCamelContext$4.call(DefaultCamelContext.java:3247)
    at org.apache.camel.impl.DefaultCamelContext$4.call(DefaultCamelContext.java:3243)
    at org.apache.camel.impl.DefaultCamelContext.doWithDefinedClassLoader(DefaultCamelContext.java:3266)
    at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:3243)
    at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:72)
    at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:3159)
    at org.apache.camel.spring.SpringCamelContext.start(SpringCamelContext.java:133)
    ... 16 common frames omitted
Caused by: org.apache.camel.FailedToCreateProducerException: Failed to create Producer for endpoint: google-pubsub://local:POC.OMEGA.CONSUMER.OUT. Reason: java.io.IOException: The Application Default Credentials are not available. They are available if running on Google App Engine, Google Compute Engine, or Google Cloud Shell. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
    at org.apache.camel.impl.ProducerCache.doGetProducer(ProducerCache.java:572)
    [...]
Caused by: java.io.IOException: The Application Default Credentials are not available. They are available if running on Google App Engine, Google Compute Engine, or Google Cloud Shell. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
    at com.google.api.client.googleapis.auth.oauth2.DefaultCredentialProvider.getDefaultCredential(DefaultCredentialProvider.java:98)
    at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.getApplicationDefault(GoogleCredential.java:217)
    at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.getApplicationDefault(GoogleCredential.java:195)
    at org.apache.camel.component.google.pubsub.GooglePubsubConnectionFactory.createDefault(GooglePubsubConnectionFactory.java:127)
    at org.apache.camel.component.google.pubsub.GooglePubsubConnectionFactory.buildClient(GooglePubsubConnectionFactory.java:101)
    at org.apache.camel.component.google.pubsub.GooglePubsubConnectionFactory.buildClient(GooglePubsubConnectionFactory.java:76)
    at org.apache.camel.component.google.pubsub.GooglePubsubConnectionFactory.getDefaultClient(GooglePubsubConnectionFactory.java:60)
    at org.apache.camel.component.google.pubsub.GooglePubsubEndpoint.afterPropertiesSet(GooglePubsubEndpoint.java:96)
    at org.apache.camel.component.google.pubsub.GooglePubsubEndpoint.createProducer(GooglePubsubEndpoint.java:104)
    at org.apache.camel.impl.ProducerCache.doGetProducer(ProducerCache.java:561)
    ... 118 common frames omitted
2020-12-03 12:18:13 INFO   | Shutting down ExecutorService 'pubSubAcknowledgementExecutor'
2020-12-03 12:18:13 INFO   | Shutting down ExecutorService 'pubsubSubscriberThreadPool'
2020-12-03 12:18:13 INFO   | Shutting down ExecutorService 'pubsubPublisherThreadPool'
2020-12-03 12:18:13 INFO   | Shutting down ExecutorService 'applicationTaskExecutor'
Disconnected from the target VM, address: '127.0.0.1:42485', transport: 'socket'

Process finished with exit code 1

如果有人已经成功地将pubsub仿真器用于camel路由，我对您的解决方案感兴趣。

Javaspring-bootgoogle-cloud-pubsubapache-camelspring-cloud-gcp

2条答案

按热度按时间
ojsjcaue

ojsjcaue1#

我对google pub/sub没有任何线索，但是错误消息说
应用程序默认凭据不可用。如果运行在googleappengine、googlecomputeengine或googlecloudshell上，就可以使用它们。否则，必须定义指向定义凭据的文件的环境变量google\u application\u credentials。
我想对于模拟器，您必须定义环境变量 GOOGLE_APPLICATION_CREDENTIALS 如错误信息所示。
还有一个google开发者页面，错误消息中提到了更多细节。
编辑
查看camel google pubsub docs，有一个端点选项称为 endpoint . 它被描述为
与本地发布/订阅仿真器一起使用的终结点。
您是否尝试将本地仿真器端点设置为

google-pubsub://project-id:destinationName?endpoint=localhost:8085
赞(0）分享  回复(0）举报  2021-07-06
4ngedf3f

4ngedf3f2#

我终于在本地连接并使用了模拟器。
第一步是添加 GOOGLE_APPLICATION_CREDENTIALS env var到我的run config（这不是很好，因为我们必须在本地获得有效的凭据才能连接到google pubsub，即使我们不使用真正的pubsub）。
然后在 application-dev.yml :
我搬走了 spring.cloud.gcp.pubsub 以及 camel.component.google-pubsub.connection-factory.serviceURL 属性
我添加了以下属性： camel.component.google-pubsub.endpoint: localhost:8085 （你说得对@burki谢谢你！）
最后（也是最重要的），我升级了camel（从2.23.2到3.6.0），我无法使我的camel应用程序与本地模拟器一起工作
注意：在启动应用程序之前，不要忘记在本地创建所需的主题/订阅：

curl -s -X PUT http://localhost:8085/v1/projects/local/topics/my-topic
curl -s -X PUT http://localhost:8085/v1/projects/local/subscriptions/my-subscription -H 'content-type: application/json' -d '{"topic":"projects/local/topics/my-topic"}'
{...}
赞(0）分享  回复(0）举报  2021-07-06
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com