我在java代码中有一个特定的例子要处理spark,但我不知道该怎么做:我想要最后一个带有partition ptf的false标记(这里的记录在desc中为列integrationdatetime排序)以及标记为true时后面的所有记录。在sql中,它看起来像:
with full_query as (
select distinct t.portfolio, t.commandUId, max(t.isPartialRelaunch) over (partition by t.commandName, t.commandUId, t.portfolio) as isPartialRelaunchPerPTF
, (DENSE_RANK() over (partition by t.portfolio, t.commandUId) order by t.i2rIntegrationTime desc) as commandUIDRank
from varFumModel t)
)
select t.*, (DENSE_RANK() over (partition by t.portfolio, t.commandUId) order by t.i2rIntegrationTime desc) as commandUIDRank from varFumModel t
where commandUIDRank <= nvl((select min(d.commandUIDRank) from full_query d where d.isPartialRelaunchPerPTF = false and d.portfolio = t.portfolio and d.commandUId = t.commandUId), 10000)
化学文摘社1
+------+------------+-------+------+---------------+-----------------------+
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position2 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
+------+------------+-------+------+---------------+-----------------------+
+------+------------+-------+------+---------------+-----------------------+
|PTF1 | Position1 | FALSE | CMD1 | COMMAND_UID_1 | integrationdatetime_t1|
+------+------------+-------+------+---------------+-----------------------+
结果
+------+------------+-------+------+---------------+-----------------------+
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position2 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position1 | FALSE | CMD1 | COMMAND_UID_1 | integrationdatetime_t1|
+------+------------+-------+------+---------------+-----------------------+
化学文摘社2
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | FALSE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position2 | FALSE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
+------+------------+-------+------+---------------+------------------------+
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | FALSE | CMD1 | COMMAND_UID_1 | integrationdatetime_t1|
+------+------------+-------+------+---------------+------------------------+
结果
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | FALSE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position2 | FALSE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
+------+------------+-------+------+---------------+------------------------+
化学文摘社3
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | FALSE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position2 | FALSE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
+------+------------+-------+------+---------------+------------------------+
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_1 | integrationdatetime_t1|
+------+------------+-------+------+---------------+------------------------+
结果
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | FALSE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position2 | FALSE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
+------+------------+-------+------+---------------+------------------------+
化学文摘社4
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position2 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
+------+------------+-------+------+---------------+------------------------+
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_1 | integrationdatetime_t1|
+------+------------+-------+------+---------------+------------------------+
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_0 | integrationdatetime_t0|
+------+------------+-------+------+---------------+------------------------+
结果
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position2 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_1 | integrationdatetime_t1|
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_0 | integrationdatetime_t0|
+------+------------+-------+------+---------------+------------------------+
化学文摘社5
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position2 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
+------+------------+-------+------+---------------+------------------------+
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_1 | integrationdatetime_t1|
+------+------------+-------+------+---------------+------------------------+
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | FALSE | CMD1 | COMMAND_UID_0 | integrationdatetime_t0|
+------+------------+-------+------+---------------+------------------------+
+------+------------+-------+------+---------------+-------------------------+
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_-1| integrationdatetime_t-1|
+------+------------+-------+------+---------------+-------------------------+
结果
+------+------------+-------+------+---------------+------------------------+
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position2 | TRUE | CMD1 | COMMAND_UID_2 | integrationdatetime_t2|
|PTF1 | Position1 | TRUE | CMD1 | COMMAND_UID_1 | integrationdatetime_t1|
|PTF1 | Position1 | FALSE | CMD1 | COMMAND_UID_0 | integrationdatetime_t0|
+------+------------+-------+------+---------------+------------------------+
暂无答案!
目前还没有任何答案,快来回答吧!