无法连接到smtp服务器:错误[no provider for smtp]scala spark

xurqigkl  于 2021-05-29  发布在  Spark
关注(0)|答案(3)|浏览(784)

代码:

object mail {

 // implicit def stringToSeq(single: String): Seq[String] = Seq(single)
  implicit def liftToOption[T](t: T): Option[T] = Some(t)

  sealed abstract class MailType
  case object Plain extends MailType
  case object Rich extends MailType
  case object MultiPart extends MailType

  case class Mail(
    from: (String, String), // (email -> name)
    to: Seq[String],
    cc: Seq[String] = Seq.empty,
    bcc: Seq[String] = Seq.empty,
    subject: String,
    message: String,
    richMessage: Option[String] = None,
    attachment: Option[(java.io.File)] = None
  )

  object send {
    def a(mail: Mail) {
      import org.apache.commons.mail._

      val format =
        if (mail.attachment.isDefined) MultiPart
        else if (mail.richMessage.isDefined) Rich
        else Plain

      val commonsMail: Email = format match {
        case Plain => new SimpleEmail().setMsg(mail.message)
        case Rich => new HtmlEmail().setHtmlMsg(mail.richMessage.get).setTextMsg(mail.message)
        case MultiPart => {
          val attachment = new EmailAttachment()
          attachment.setPath(mail.attachment.get.getAbsolutePath)
          attachment.setDisposition(EmailAttachment.ATTACHMENT)
          attachment.setName(mail.attachment.get.getName)
          new MultiPartEmail().attach(attachment).setMsg(mail.message)
        }
      }

      // TODO Set authentication from your configuration, sys properties or w/e
      commonsMail.setHostName("smtp.office365.com")
      commonsMail.setSmtpPort(587)
      commonsMail.setAuthentication("someUserName", "TestPassword")
      commonsMail.setStartTLSEnabled(true)
      // Can't add these via fluent API because it produces exceptions
      mail.to foreach (commonsMail.addTo(_))
      mail.cc foreach (commonsMail.addCc(_))
      mail.bcc foreach (commonsMail.addBcc(_))

      commonsMail.
        setFrom(mail.from._1, mail.from._2).
        setSubject(mail.subject).
        send()
    }
  }
}

我得到以下错误:

0/06/17 05:22:32 ERROR ApplicationMaster: User class threw exception: org.apache.commons.mail.EmailException: Sending the email to the following server failed : smtp.office365.com:587
org.apache.commons.mail.EmailException: Sending the email to the following server failed : smtp.office365.com:587
        at org.apache.commons.mail.Email.sendMimeMessage(Email.java:1421)
        at org.apache.commons.mail.Email.send(Email.java:1448)
        at mail$send$.a(mail.scala:56)
        at ImpressionThrottlingPattern$.main(pattern.scala:292)
        at ImpressionThrottlingPattern.main(pattern.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:721)
Caused by: javax.mail.NoSuchProviderException: No provider for smtp
        at javax.mail.Session.getProvider(Session.java:479)
        at javax.mail.Session.getTransport(Session.java:681)
        at javax.mail.Session.getTransport(Session.java:662)
        at javax.mail.Session.getTransport(Session.java:719)
        at javax.mail.Transport.send0(Transport.java:248)
        at javax.mail.Transport.send(Transport.java:124)
        at org.apache.commons.mail.Email.sendMimeMessage(Email.java:1411)
        ... 9 more

我试图使用telnet检查我正在使用的smtp服务器是否处于活动状态。我发现它很活跃:

[root@qa-mukund mukund.sudharsan]# telnet smtp.office365.com 587
Trying xx.xx.xx.xxx...
Connected to smtp.office365.com.
Escape character is '^]'.
220 BY5PR04CA0025.outlook.office365.com Microsoft ESMTP MAIL Service ready at Wed, 17 Jun 2020 13:48:54 +0000

这是我的sbt文件:

name := "TestMukund"
version := "1.0"
scalaVersion := "2.11.8"
val sparkVersion = "2.3.0"
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.3.0",
"org.apache.spark" %% "spark-sql" % "2.3.0",
"com.databricks" %% "spark-csv" % "1.5.0",
"com.typesafe" % "config" % "1.2.0",
"mysql" % "mysql-connector-java" % "5.1.12",
"org.apache.commons" % "commons-dbcp2" % "2.0.1",
"org.apache.commons" % "commons-email" % "1.4"
)

assemblyMergeStrategy in assembly := {
 case PathList("META-INF", xs @ _*) => MergeStrategy.discard
 case x => MergeStrategy.first
}

我不认为问题出在用户名和密码上,因为错误表明找不到smtp提供程序本身。我对scala和堆栈溢出非常陌生。我希望以上细节足以找到问题的根源。
通过以下命令执行编辑应用程序:

spark-submit --master yarn-cluster 
             --class ImpressionThrottlingPattern 
             --name Test_MukundDBRead 
             /home/ml.user/mukund/prod/ImpressionThrottlingPatternGenerator/target/scala-2.11/TestMukund-assembly-1.0.jar

我尝试通过将sbt文件更新为以下内容来克服javax.mail-api verision的1.6.2版本:

libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.3.0",
"org.apache.spark" %% "spark-sql" % "2.3.0",
"com.typesafe" % "config" % "1.2.0",
"mysql" % "mysql-connector-java" % "5.1.12",
"org.apache.commons" % "commons-dbcp2" % "2.0.1",
"org.apache.commons" % "commons-email" % "1.5",
)
dependencyOverrides ++= Seq(
        "javax.activation" % "activation" % "1.1.1",
        "javax.mail" % "javax.mail-api" % "1.6.2"
)

assemblyMergeStrategy in assembly := {
 case PathList("META-INF", xs @ _*) => MergeStrategy.discard
 case x => MergeStrategy.last
}

但效果不太好。
我尝试运行sbt executed命令,发现输出包含以下行(不确定这是否有用):

[info] Here are other dependency conflicts that were resolved:
[info]  * javax.activation:activation:1.1.1 is selected over 1.1
[info]      +- org.apache.commons:commons-email:1.4               (depends on 1.1.1)
[info]      +- com.sun.mail:javax.mail:1.5.2                      (depends on 1.1)
[success] Total time: 4 s, completed Jun 18, 2020 1:10:36 AM
pokxtpni

pokxtpni1#

错误是因为代码无法找到/加载任何java类实现来与之通信 smtp 服务器。没有这样的例外
应该有多个或旧版本的 javamail 或者 activation 运行时类路径中的jar。确保你摆脱这些,并有最新的版本
为了检查sbt的依赖树, sbt "inspect tree clean"

eaf3rand

eaf3rand2#

经过一番努力,我终于解决了这个问题,
我必须下载邮件和激活jar,并通过spark submit命令显式传递。喜欢

spark-submit --jars <path_to_activation_jar>,<path_to_mail_jar> <other parameters>

将sbt文件中的configs(注意从前面删除%%)更改为以下内容:

val SparkVersion = "2.3.2"
scalaVersion := "2.11.8"

libraryDependencies ++= Seq(
"org.apache.spark" % "spark-core_2.11" % SparkVersion,
"org.apache.spark" % "spark-sql_2.11" % SparkVersion,
"com.typesafe" % "config" % "1.2.0",
"mysql" % "mysql-connector-java" % "5.1.12",
"org.apache.commons" % "commons-dbcp2" % "2.0.1",
"org.apache.commons" % "commons-email" % "1.4",
)

然而,我仍然想知道如何通过sbt程序集自动下载邮件和激活jar而不显式地传递。

31moq8wy

31moq8wy3#

尝试在build.sbt中使用下面的库来修复此问题。

"org.apache.commons" % "commons-email" % "1.5"

相关问题