我已经创建了一个有3个代理的AWS MSK集群。集群的身份验证是使用IAM role-based authentication
和SASL/SCRAM authentication
。我已经在我的微服务中使用了MSK,并且我在代理中发送和接收事件。现在我想根据Kafka主题的事件计数来扩展我的Kubernetes pod。
我有一个包含多个微服务的EKS集群,这些微服务的名称空间称为default
。
我已经在一个名为keda
的新名称空间中安装了KEDA。我还向KEDA添加了一个TriggerAuthentication
,并将其安装在default
名称空间中。我使用SASL/SCRAM authentication
通过MSK对KEDA进行了身份验证:
apiVersion: v1
kind: Secret
metadata:
name: keda-kafka-secrets
namespace: {{ .Release.Namespace }}
data:
username: "{{ $.Values.kafka.username | b64enc }}"
password: "{{ $.Values.kafka.password | b64enc }}"
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: keda-trigger-auth-kafka-conn
namespace: {{ .Release.Namespace }}
spec:
secretTargetRef:
- parameter: username
name: keda-kafka-secrets
key: username
- parameter: password
name: keda-kafka-secrets
key: password
字符串
我将ScaledObject
添加到了我的微服务的Helm Charts中:
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: user-service-scaledobject
spec:
scaleTargetRef:
kind: Deployment
name: {{ $.Chart.Name }}
pollingInterval: 30 # Optional. Default: 30 seconds
cooldownPeriod: 300 # Optional. Default: 300 seconds
idleReplicaCount: 0 # Optional. Default: 0
minReplicaCount: 0 # Optional. Default: 0
maxReplicaCount: 100 # Optional. Default: 100
triggers:
- type: kafka
authenticationRef:
name: keda-trigger-auth-kafka-conn
metadata:
bootstrapServers: {{ .Values.kafka.bootstrapServers }}
consumerGroup: {{ .Values.kafka.consumerGroup }}
topic: {{ .Values.kafka.topic }}
lagThreshold: "500"
version: 2.8.1
sasl: scram_sha512
型
这是微服务的Deployment
:
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ $.Chart.Name }}
labels:
app: {{ $.Chart.Name }}
component: {{ $.Chart.Name }}
release: {{ $.Release.Name }}
spec:
replicas: {{ .Values.deployment.replicas }}
strategy:
type: RollingUpdate
rollingUpdate:
{{- if .Values.deployment.rollingUpdate.enabled }}
maxSurge: {{ $.Values.deployment.rollingUpdate.maxSurge }}
maxUnavailable: {{ $.Values.deployment.rollingUpdate.maxUnavailable }}
{{- end }}
selector:
matchLabels:
app: {{ $.Chart.Name }}
release: {{ $.Release.Name }}
template:
metadata:
labels:
app: {{ $.Chart.Name }}
version: {{ $.Chart.Version }}
release: {{ $.Release.Name }}
spec:
imagePullSecrets:
- name: {{ $.Values.deployment.image.imagepullsecret }}
containers:
- name: {{ $.Chart.Name }}
image: "{{ .Values.deployment.image.repository }}:{{ .Values.deployment.image.tag }}"
ports:
- name: {{ $.Chart.Name }}
containerPort: {{ $.Values.deployment.ports.http }}
protocol: {{ $.Values.service.protocol }}
imagePullPolicy: {{ $.Values.deployment.image.pullPolicy | quote }}
resources:
requests:
memory: "{{ .Values.deployment.resources.requests.memory }}"
cpu: "{{ .Values.deployment.resources.requests.cpu }}"
limits:
memory: "{{ .Values.deployment.resources.limits.memory }}"
cpu: "{{ .Values.deployment.resources.limits.cpu }}"
{{- if $.Values.deployment.volumes.enabled }}
volumeMounts:
- name: volume
mountPath: /var/volume/myapp
{{- end }}
{{- if $.Values.deployment.env }}
env:
{{- range .Values.deployment.env }}
- name: {{ .name }}
value: "{{ .value }}"
{{- end }}
{{- end }}
{{- if $.Values.deployment.volumes.enabled }}
volumes:
- name: volume
emptyDir: {}
{{- end }}
{{- if $.Values.deployment.nodeSelector }}
nodeSelector:
{{- toYaml $.Values.deployment.nodeSelector | nindent 8 }}
{{- end }}
型
当我尝试将ScaledObject
添加到微服务名称空间(default
)时,我在KEDA操作员窗格中收到以下错误:
ERROR scale_handler error resolving auth params
ERROR Error getting scalers
ERROR Failed to create new HPA resource
Warning KEDAScalerFailed 13m (x82 over 18h) keda-operator error creating kafka client: kafka: client has run out of available brokers to talk to
Warning ScaledObjectCheckFailed 13m (x82 over 18h) keda-operator failed to ensure HPA is correctly created for ScaledObject
型
有什么问题吗?
谢谢你,谢谢
1条答案
按热度按时间sh7euo9m1#
这可能有点晚,但根据KEDA的文档,实际上有两个Kafka缩放器:
只有Experimental flavor支持AWS MSK IAM身份验证。如果您查看两者的触发器规范,您会注意到
type
的不同。apache-kafka
是新的实验版本。字符串
请参阅PR了解更多详情:https://github.com/kedacore/keda/pull/4801
也许这种微妙的差异就是两个版本的定标器之间的差异。