Go语言 在SQS和内存泄漏https使用上接收消息时,I/O超时问题

x0fgdtte  于 2023-03-21  发布在  Go
关注(0)|答案(1)|浏览(95)

AWS SDK版本:v1.38.19
Go版本:go1.15.7
阿尔卑斯山3.7
我正在使用标准队列,我正在应用程序中立即初始化SQS连接,如下所示;

// Connection connection to the SQS
var Connection *sqs.SQS

// InitSQS initialize the AWS SQS connection
func InitSQS() {
    sess := session.Must(session.NewSessionWithOptions(session.Options{
        SharedConfigState: session.SharedConfigEnable,
    }))

    Connection = sqs.New(sess, &aws.Config{
        Region:     aws.String("eu-west-1"),
        DisableSSL: aws.Bool(true),
    })
}

我禁用SSL是因为;我有内存和CPU泄漏时,我去与SSL在我的应用程序(我的应用程序是不开放的世界其他地方顺便说一句,它是一个内部服务,我的其他应用程序)。
下面是我用来从SQS读取消息的配置:

func ConsumeUpdateMessage(db *database.MySQLWrap, sqsApi queue.SQSAPI) error {
    result, err := sqsApi.ReceiveMessage(&sqs.ReceiveMessageInput{
        AttributeNames: []*string{
            aws.String(sqs.MessageSystemAttributeNameSentTimestamp),
        },
        MessageAttributeNames: []*string{
            aws.String(sqs.QueueAttributeNameAll),
        },
        QueueUrl:            &qURL,
        MaxNumberOfMessages: aws.Int64(10),
        WaitTimeSeconds:     aws.Int64(20),
    })

    if err != nil {
        return fmt.Errorf("error on receiving the message from queue: %s", err.Error())
    }

    for _, msg := range result.Messages {
            // business logic
        }

    return err
}

这就是我调用ConsumeUpdateMessage方法的方式

// InitializeUpdateMessage ..
func InitializeUpdateMessage(db *database.MySQLWrap, sqsApi queue.SQSAPI) {
    go func() {
        for {
            time.Sleep(500 * time.Millisecond)

            err := ConsumeUpdateMessage(db, sqsApi)

            if err != nil {
                log.Error(err)

                continue
            }
        }
    }()
}

但有时我的订阅者会返回这样的错误

*awserr.baseError: RequestError: send request failed
caused by: Post "http://sqs.eu-west-1.amazonaws.com/": dial tcp xx.x.xx.xxx:80: i/o timeout

(Note:我输入xx而不是共享IP地址)
我在论坛和其他地方找了找,但我找不到解决这两个问题的方法,
1.使用SSL连接时出现内存泄漏(注意:但由于我的应用程序是一个内部服务,因此我认为不必使用SSL)
1.输入/输出超时

hyrbngr7

hyrbngr71#

这是一个很晚的答案,但我遇到了类似的问题,我想我发现了问题是什么:您为长轮询设置了20秒的WaitTimeSeconds(这是一个好主意),但是HTTP客户机超时的时间比这要早。
WaitTimeSeconds的文档说明:

// To avoid HTTP errors, ensure that the HTTP response timeout for ReceiveMessage
// requests is longer than the WaitTimeSeconds parameter. For example, with
// the Java SDK, you can set HTTP transport settings using the NettyNioAsyncHttpClient
// (https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/http/nio/netty/NettyNioAsyncHttpClient.html)
// for asynchronous clients, or the ApacheHttpClient (https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/http/apache/ApacheHttpClient.html)
// for synchronous clients.

至少,从v1.44.39aws-sdk-go
下一页介绍了如何配置自定义HTTP客户机:https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/custom-http.html
我将把该页的所有代码复制到这里,以供后人使用:

type HTTPClientSettings struct {
    Connect          time.Duration
    ConnKeepAlive    time.Duration
    ExpectContinue   time.Duration
    IdleConn         time.Duration
    MaxAllIdleConns  int
    MaxHostIdleConns int
    ResponseHeader   time.Duration
    TLSHandshake     time.Duration
}

func NewHTTPClientWithSettings(httpSettings HTTPClientSettings) (*http.Client, error) {
    var client http.Client
    tr := &http.Transport{
        ResponseHeaderTimeout: httpSettings.ResponseHeader,
        Proxy:                 http.ProxyFromEnvironment,
        DialContext: (&net.Dialer{
            KeepAlive: httpSettings.ConnKeepAlive,
            DualStack: true,
            Timeout:   httpSettings.Connect,
        }).DialContext,
        MaxIdleConns:          httpSettings.MaxAllIdleConns,
        IdleConnTimeout:       httpSettings.IdleConn,
        TLSHandshakeTimeout:   httpSettings.TLSHandshake,
        MaxIdleConnsPerHost:   httpSettings.MaxHostIdleConns,
        ExpectContinueTimeout: httpSettings.ExpectContinue,
    }

    // So client makes HTTP/2 requests
    err := http2.ConfigureTransport(tr)
    if err != nil {
        return &client, err
    }

    return &http.Client{
        Transport: tr,
    }, nil
}

httpClient, err := NewHTTPClientWithSettings(HTTPClientSettings{
    Connect:          5 * time.Second,
    ExpectContinue:   1 * time.Second,
    IdleConn:         90 * time.Second,
    ConnKeepAlive:    30 * time.Second,
    MaxAllIdleConns:  100,
    MaxHostIdleConns: 10,
    ResponseHeader:   5 * time.Second,
    TLSHandshake:     5 * time.Second,
})
if err != nil {
    fmt.Println("Got an error creating custom HTTP client:")
    fmt.Println(err)
    return
}

sess := session.Must(session.NewSession(&aws.Config{
    HTTPClient: httpClient,
}))

sqsClient := sqs.New(sess)

您将需要配置超时设置。特别是,我将ConnectTLSHandshakeResponseHeader设置为25秒,不再收到任何错误。

相关问题