一个启用了执行器的spring-boot嵌入式tomcatweb应用在部署到kubernetes时会从tomcat获得404响应

brjng4g3  于 2023-08-04  发布在  Spring
关注(0)|答案(1)|浏览(133)

我有一个在k8中运行的spring Boot web应用程序的问题,这个问题在我的本地dev上运行时是不可复制的,并且配置完全相同-只有在k8中运行时,我才得到一个404命中http://localhost:8080/actuator。
配置摘要:

  • spring-boot-starter版本2.7.10。
  • tomcatweb服务器提供的唯一端点是执行器端点集。
  • k8上的java版本是1.8.0_345;本地开发机器1.8.0_151
  • 应用程序.yaml*
server:
  port: 8080
  error:
    whitelabel:
      enabled: true
    include-exception: true
    include-message: always
    include-binding-errors: always
    include-stacktrace: always
  servlet:
    context-path: '/'

logging:
  level:
    org:
      apache:
        tomcat: DEBUG
        catalina: DEBUG
      springframework:
        web:
          filter:
            CommonsRequestLoggingFilter: DEBUG

# actuator
management:
  endpoints:
    enabled-by-default: true
    web.exposure.include: "*"
  endpoint:
    shutdown.enabled: false
    info.enabled: true


hikariCP:
  dataSourceClassName: com.microsoft.sqlserver.jdbc.SQLServerDataSource

spring:
  # Preferred JSON mapper to use for HTTP message conversion.
  mvc.converters.preferred-json-mapper: gson
  autoconfigure:
    exclude: org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration
  datasource:
    bet:
      url: someserver
      username: MSSQL_USER
      password: MSSQL_PASS
      hikari:
        poolName: betHikariCP
        maximum-pool-size: 5
        connectionTimeout: 30000
        idleTimeout: 600000
        maxLifetime: 1800000

字符串
观察到的行为:

  • 在本地运行,我可以在localhost:8080/actuator上命中执行器,但在kubernetes上运行相同的配置,当我执行到pod并运行curl localhost:8080/actuator时,我得到404错误消息- * 注意此响应来自Apache Tomcat/9.0.73*

404响应

<html lang="en">
        <head>
            <title>HTTP Status 404 – Not Found</title>
            <style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style>
        </head>
        <body>
            <h1>HTTP Status 404 – Not Found</h1>
            <hr class="line"/>
            <p>
                <b>Type</b> Status Report</p>
            <p>
                <b>Description</b> The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.</p>
            <hr class="line"/>
            <h3>Apache Tomcat/9.0.73</h3>
        </body>
    </html>

  • 本地和kubernetes上的日志输出表明
  • tomcat启动并绑定到端口8080,没有错误:Tomcat started on port(s): 8080 (http) with context path ''
  • 致动器端点竖立。记录仪org.springframework.boot.actuate.endpoint.web.EndpointLinksResolver输出Exposing 13 endpoint(s) beneath base path '/actuator'
  • 我已经配置了CommonsRequestLoggingFilter,它将记录所有请求;在本地,当我在本地点击/actuator时,它会输出一条语句,但在kubernetes上它不会输出任何东西。

输出(仅本地-在k8上我什么也得不到):

2023-07-14 10:25:23.610 DEBUG PRE-REQUEST: GET /actuator]
2023-07-14 10:25:23.656 DEBUG POST-REQUEST: GET /actuator]


还请注意,我可以在本地输入一个错误的路由,例如。actuatorxxx和我仍然得到PRE和POST日志输出,我在本地得到的404响应是json而不是html:

{
    "timestamp": "Jul 14, 2023 10:31:03 AM",
    "status": 404,
    "error": "Not Found",
    "message": "No message available",
    "path": "/actuatorxxx"
}


CommonsRequestLoggingFilter配置

@Bean
    public CommonsRequestLoggingFilter logFilter() {
        CommonsRequestLoggingFilter filter = new CommonsRequestLoggingFilter();
        filter.setIncludeQueryString(true);
        filter.setIncludeHeaders(false);
        filter.setBeforeMessagePrefix("PRE-REQUEST: ");
        filter.setAfterMessagePrefix("POST-REQUEST: ");
        return filter;
    }

  1. tomcat/ Catalina 调试日志记录、请求日志记录和配置服务器/错误页面以返回堆栈跟踪等没有给出路由/执行器未被解析的原因的任何细节
    1.日志中也没有任何异常
    还请注意,当我执行到pod并使用curl时,我相当肯定这与k8中的ngnix/ingress配置无关。
    经消毒的豆荚舱单提取物
kind: Pod
apiVersion: v1
metadata:
  name: my-app-sit-0
  generateName: my-app-sit-
  namespace: tdt-sit
  uid: d7f346e1-cbd7-4543-b4df-6da61defedae
  resourceVersion: '1263848088'
  creationTimestamp: '2023-07-08T14:18:33Z'
  labels:
    app: my-app-sit
    controller-revision-hash: my-app-sit-7d457f7dcb
    release: my-app-sit
    statefulset.kubernetes.io/pod-name: my-app-sit-0
  annotations:
    kubernetes.io/psp: eks.privileged
    my-app.com/gitlab-project-id: '6778'
    my-app.com/gitlab-project-url: https://gitlab.my-app.com./my-app-deploy
spec:
  volumes:
    - name: aws-iam-token
      projected:
        sources:
          - serviceAccountToken:
              audience: sts.amazonaws.com
              expirationSeconds: 86400
              path: token
        defaultMode: 420
    - name: my-app-sit-mount
      persistentVolumeClaim:
        claimName: my-app-sit-mount-my-app-sit-0
    - name: secret-my-app-bqsakeys
      secret:
        secretName: my-app-bqsakeys
        defaultMode: 420
    - name: kube-api-access-7gvf6
      projected:
        sources:
          - serviceAccountToken:
              expirationSeconds: 3607
              path: token
          - configMap:
              name: kube-root-ca.crt
              items:
                - key: ca.crt
                  path: ca.crt
          - downwardAPI:
              items:
                - path: namespace
                  fieldRef:
                    apiVersion: v1
                    fieldPath: metadata.namespace
        defaultMode: 420
  containers:
    - name: my-app-container
      image: >-
        someimage
      command:
        - /bin/bash
        - '-c'
      args:
        - >-
          /opt/my-app-configure-n-start.sh && tail -F
          /opt/logs/my-app.log
      ports:
        - containerPort: 80
          protocol: TCP
      envFrom:
        - secretRef:
            name: my-app-credentials
      resources:
        limits:
          memory: 4Gi
        requests:
          cpu: 500m
          memory: 4Gi
      volumeMounts:
        - name: my-app-sit-mount
          mountPath: /mnt
        - name: kube-api-access-7gvf6
          readOnly: true
          mountPath: /var/run/secrets/kubernetes.io/serviceaccount
        - name: aws-iam-token
          readOnly: true
          mountPath: /var/run/secrets/eks.amazonaws.com/serviceaccount
      livenessProbe:
        exec:
          command:
            - /bin/sh
            - /opt/aux-scripts/my-app-liveness-check.sh
            - '8558'
        initialDelaySeconds: 180
        timeoutSeconds: 5
        periodSeconds: 10
        successThreshold: 1
        failureThreshold: 3
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      imagePullPolicy: IfNotPresent
      securityContext:
        capabilities:
          add:
            - SYS_ADMIN
            - DAC_READ_SEARCH
  restartPolicy: Always
  terminationGracePeriodSeconds: 30
  dnsPolicy: ClusterFirst
  serviceAccountName: my-app-sit
  serviceAccount: my-app-sit
  nodeName: ip-xxx-xxx-xxx-xxx.my-app-node
  securityContext: {}
  imagePullSecrets:
    - name: regsecret
  hostname: my-app-sit-0
  subdomain: my-app-sit-svc


app-configure-n-start.sh脚本:

$APP_HOME/scripts/backgrounded-start > /dev/null 2> $APP_HOME/stderr.out &


上面的注解在后台运行启动脚本
启动脚本:

JAVA_OPTS="$APP_OTEL_AGENT_JAVA_COMMAND \
                                  $APP_JVM_HEAP_MIN $APP_JVM_HEAP_MAX -Xloggc:$APP_HOME/logs/gc.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps\
                                  -XX:-PrintTenuringDistribution -XX:+PrintGCCause -XX:+PrintGCApplicationStoppedTime -XX:+UseGCLogFileRotation\
                                  -XX:NumberOfGCLogFiles=15 -XX:GCLogFileSize=2M -XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError\
                                  -XX:HeapDumpPath=$APP_HOME/logs/ -XX:+DisableExplicitGC"

AKKA_CLASSPATH="$APP_HOME/lib/*:$APP_HOME/bin/*:$APP_HOME/config:$APP_HOME/lib/sigar/*:$APP_HOME/../*"

java $JAVA_OPTS -cp "$AKKA_CLASSPATH" -Dspring.profiles.active=<active-profile> au.com.my.app.akka.SpringApplication --spring.main.banner-mode=off


谁能建议我添加额外的日志来说明发生了什么?或者可能的原因?

bvhaajcl

bvhaajcl1#

这个问题的根本原因是,我在本地运行 * 没有 * otel代理,而在k8 deploy $APP_OTEL_AGENT_JAVA_COMMAND中,我在java命令中添加了以下内容:
第一个月
2021年起使用的otel版本为1.7.0;而且它干扰了tomcat/spring Boot 。
升级到最新的otel 1.28.0解决了这个问题。

相关问题