我试图读取一个apache pig拉丁语文本文件,该文件的每一行都包含非分隔ascii。也就是说,该行中的每一列在该行的特定位置开始和结束。
样本定义:
+--------+----------------+--------------+
| Column | Start Position | End Position |
+--------+----------------+--------------+
| A | 1 | 6 |
+--------+----------------+--------------+
| B | 8 | 11 |
+--------+----------------+--------------+
| C | 13 | 15 |
+--------+----------------+--------------+
样本数据:
+---+---+---+---+---+---+---+----+---+----+----+----+----+----+----+
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 |
+---+---+---+---+---+---+---+----+---+----+----+----+----+----+----+
| s | a | m | p | l | e | | d | a | t | a | | | h | i |
+---+---+---+---+---+---+---+----+---+----+----+----+----+----+----+
| d | u | d | e | | | | hi | | | | | b | r | o |
+---+---+---+---+---+---+---+----+---+----+----+----+----+----+----+
预期产量:
sample, data, hi
dude, hi, bro
我怎么读这个在Pig?pigstorage似乎不够灵活,不允许位置分隔,只允许字符串分隔(逗号、制表符等)。
1条答案
按热度按时间nbysray51#
看起来apache为这个特定用例提供了一个加载程序:
https://pig.apache.org/docs/r0.16.0/api/