pyspark 将字符串列转换为日期格式时获取空值

yeotifhr  于 2023-10-15  发布在  Spark
关注(0)|答案(1)|浏览(112)

我得到空值的日期这里是我的代码。我已经创建了一个字符串格式和日期传递

data=[('anmol','',27000,'21/01/1998'),('anubhav',25,39000,'20/06/1997'),('ravi',25,8900,''),              ('','',70000,''),('','','','')]`
    from pyspark.sql.types import StringType, IntegerType, StructType, StructField
    schema=StructType([StructField('Name', StringType()),
            StructField('Age', StringType()),
            StructField('Income', StringType()),
            StructField('Date', StringType())])
    from pyspark.sql import SparkSession
    spark = SparkSession.builder.appName("Python Spark SQL basic example")\
    .config("spark.some.config.option", "some-value").getOrCreate()
    df1=spark.createDataFrame(data=data, schema=schema)

这里我正在把字符串日期格式转换成日期格式
df3=df1.select(to_date(df1.Date,'yyyy-MM-dd ').alias('dt'))

qgelzfjb

qgelzfjb1#

尝试更改数据格式-

df3=df1.select(to_date(df1.Date,'dd/MM/yyyy').alias('dt'))

相关问题