我正在尝试通过livy使用交互式会话读取文件。我创建了一个会话并提交了一个要执行的语句。语句执行得很好,但响应在一定长度后被截断。例如,这是我的回答get:-
{
"id": 17,
"code": "import org.apache.spark.sql.types.StringType; val dataframe = new org.apache.spark.sql.SQLContext(sc).read.parquet(\"/ClickstreamInput\"); dataframe.select(dataframe.columns.map(c => col(c).cast(StringType)): _*).na.fill(\"null\").toJSON.rdd.collect().toList.map(_.toString)",
"state": "available",
"output": {
"status": "ok",
"execution_count": 17,
"data": {
"text/plain": "warning: there was one deprecation warning; re-run with -deprecation for details\nimport org.apache.spark.sql.types.StringType\ndataframe: org.apache.spark.sql.DataFrame = [value: string]\nres16: List[String] = List({\"value\":\"m6pHQD5Qf0HWiLI,2254023209288530,2069219912,product_view,facebook,desktop\"}, {\"value\":\"pwSyumNQcy0o3s1,2253999803834925,2081337593,product_view,facebook,ios\"}, {\"value\":\"KVsFm9dd49TlmGA,2253999782497840,533382031,product_view,google,android\"}, {\"value\":\"y6AgiVTOpqE4l5q,2253999749428795,533382031,product_view,facebook,android\"}, {\"value\":\"XXKYip1uDMeUXF1,2254032037691786,1174233544,purchase,facebook,desktop\"}, {\"value\":\"KJ7nUudfdXUMMdB,2254024802941761,1585974151,cart_add,google,android\"}, {\"value\":\"vRB1O17qA1sE0JL,2253999779673921,571768064,product_view,instagram,android\"}, {\"value\":\"Bw2R1MuAGNBuG31,2254009356680570,1773864570,product_view,faceboo..."
}
},
"progress": 1.0
}
有没有办法增加这个字符限制并得到完整的响应?
暂无答案!
目前还没有任何答案,快来回答吧!