我正在研究cca-175练习题。我得到了一个文本文件，它被 | :
Christopher|Jan 11, 2015, |5
Kapil|11 Jan, 2015|5
Thomas|6/17/2014|5
John|22-08-2013|5
Mithun|2013|5
Jitendra||5然后我将文件保存为rdd并尝试Map它。但是，当在中使用单引号和双引号时 split 方法，scala返回两个不同的结果，使用单引号是正确的。
使用单引号 line.split('|') ，它返回： Array[String] = Array(Christopher, Jan 11, 2015, 5) ，这是对的。
使用双引号 line.split("|") ，它返回： Array[String] = Array(C, h, r, i, s, t, o, p, h, e, r, |, J, a, n, " ", 1, 1, , " ", 2, 0, 1, 5, |, 5) ，这不是我需要的。
有人能帮我回答这个问题吗？谢谢！
scala> val feedbackmap = feedback.map(line=>line.split('|'))
feedbackmap: org.apache.spark.rdd.RDD[Array[String]] = MapPartitionsRDD[4] at map at <console>:29
scala> feedbackmap.first
19/04/10 14:15:55 INFO SparkContext: Starting job: first at <console>:32
19/04/10 14:15:55 INFO DAGScheduler: Got job 4 (first at <console>:32) with 1 output partitions
19/04/10 14:15:55 INFO DAGScheduler: Final stage: ResultStage 4 (first at <console>:32)
19/04/10 14:15:55 INFO DAGScheduler: Parents of final stage: List()
19/04/10 14:15:55 INFO DAGScheduler: Missing parents: List()
19/04/10 14:15:55 INFO DAGScheduler: Submitting ResultStage 4 (MapPartitionsRDD[4] at map at <console>:29), which has no missing parents
19/04/10 14:15:55 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 3.4 KB, free 510.7 MB)
19/04/10 14:15:55 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 2003.0 B, free 510.7 MB)
19/04/10 14:15:55 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on localhost:43371 (size: 2003.0 B, free: 511.1 MB)
19/04/10 14:15:55 INFO SparkContext: Created broadcast 5 from broadcast at DAGScheduler.scala:1008
19/04/10 14:15:55 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 4 (MapPartitionsRDD[4] at map at <console>:29)
19/04/10 14:15:55 INFO TaskSchedulerImpl: Adding task set 4.0 with 1 tasks
19/04/10 14:15:55 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 5, localhost, partition 0,ANY, 2171 bytes)
19/04/10 14:15:55 INFO Executor: Running task 0.0 in stage 4.0 (TID 5)
19/04/10 14:15:55 INFO HadoopRDD: Input split: hdfs://nn01.itversity.com:8020/user/junyanxu/scenario_37/feedback.txt:0+58
19/04/10 14:15:55 INFO Executor: Finished task 0.0 in stage 4.0 (TID 5). 2173 bytes result sent to driver
19/04/10 14:15:55 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 5) in 7 ms on localhost (1/1)
19/04/10 14:15:55 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool
19/04/10 14:15:55 INFO DAGScheduler: ResultStage 4 (first at <console>:32) finished in 0.007 s
19/04/10 14:15:55 INFO DAGScheduler: Job 4 finished: first at <console>:32, took 0.012483 s
19/04/10 14:15:55 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool
res3: Array[String] = Array(Christopher, Jan 11, 2015, 5)
scala> 19/04/10 14:20:55 WARN SparkContext: Killing executors is only supported in coarse-grained mode
19/04/10 14:20:55 WARN ExecutorAllocationManager: Unable to reach the cluster manager to kill executor driver!
val
scala> val feedbackmap2 = feedback.map(line=>line.split("|"))
feedbackmap2: org.apache.spark.rdd.RDD[Array[String]] = MapPartitionsRDD[5] at map at <console>:29
scala> feedbackmap2.first
19/04/10 14:22:58 INFO SparkContext: Starting job: first at <console>:32
19/04/10 14:22:58 INFO DAGScheduler: Got job 5 (first at <console>:32) with 1 output partitions
19/04/10 14:22:58 INFO DAGScheduler: Final stage: ResultStage 5 (first at <console>:32)
19/04/10 14:22:58 INFO DAGScheduler: Parents of final stage: List()
19/04/10 14:22:58 INFO DAGScheduler: Missing parents: List()
19/04/10 14:22:58 INFO DAGScheduler: Submitting ResultStage 5 (MapPartitionsRDD[5] at map at <console>:29), which has no missing parents
19/04/10 14:22:58 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 3.4 KB, free 510.7 MB)
19/04/10 14:22:58 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 2003.0 B, free 510.7 MB)
19/04/10 14:22:58 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on localhost:43371 (size: 2003.0 B, free: 511.1 MB)
19/04/10 14:22:58 INFO SparkContext: Created broadcast 6 from broadcast at DAGScheduler.scala:1008
19/04/10 14:22:58 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 5 (MapPartitionsRDD[5] at map at <console>:29)
19/04/10 14:22:58 INFO TaskSchedulerImpl: Adding task set 5.0 with 1 tasks
19/04/10 14:22:58 INFO TaskSetManager: Starting task 0.0 in stage 5.0 (TID 6, localhost, partition 0,ANY, 2171 bytes)
19/04/10 14:22:58 INFO Executor: Running task 0.0 in stage 5.0 (TID 6)
19/04/10 14:22:58 INFO HadoopRDD: Input split: hdfs://nn01.itversity.com:8020/user/junyanxu/scenario_37/feedback.txt:0+58
19/04/10 14:22:58 INFO Executor: Finished task 0.0 in stage 5.0 (TID 6). 2244 bytes result sent to driver
19/04/10 14:22:58 INFO TaskSetManager: Finished task 0.0 in stage 5.0 (TID 6) in 12 ms on localhost (1/1)
19/04/10 14:22:58 INFO TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool
19/04/10 14:22:58 INFO DAGScheduler: ResultStage 5 (first at <console>:32) finished in 0.012 s
19/04/10 14:22:58 INFO DAGScheduler: Job 5 finished: first at <console>:32, took 0.040166 s
res4: Array[String] = Array(C, h, r, i, s, t, o, p, h, e, r, |, J, a, n, " ", 1, 1, ,, " ", 2, 0, 1, 5, |, 5)
2条答案
按热度按时间ct3nt3jp1#
在scala中，单引号表示字符，因此split（“|”）使用|字符。当您使用双引号时，您使用了一个字符串，而且split可以接受regex字符串，因此字符串中未转义的|被解释为regex或
t5zmwmid2#
我认为阿农·罗特姆·加洛兹对
|在字符串内部作为参数进行拆分：它是一个逻辑运算符。此外，这里发生的是使用regex，这意味着空字符串或空字符串。因为空字符串基本上可以在
String（如果有帮助的话，你可以理解"ab"相当于"a" + "" + "b")，在每个字符之间进行拆分。另请参见scala string.split不起作用的状态：
如果你使用
split('|')或者split("""\|""")你应该得到你想要的。事实上，一个逃犯
|不再被认为是逻辑运算符，而是正则表达式中的字符本身。