Update from Sync Service
This commit is contained in:
@@ -0,0 +1,22 @@
|
||||
## 1. 问题
|
||||
|
||||
在hive中使用了过多的窗口函数导致了内存OOM
|
||||
|
||||
## 2. 原因
|
||||
|
||||
每一个窗口函数,hive都会在内存中执行一次窗口划分,消耗大量的内存,最终导致OOM;
|
||||
|
||||
## 3. 解决方法
|
||||
|
||||
使用window算子,复用窗口,如:
|
||||
```sql
|
||||
select
|
||||
id,
|
||||
ts,
|
||||
v,
|
||||
lead(v) over w as lead_v,
|
||||
lag(v) over w as lag_v
|
||||
from
|
||||
test
|
||||
WINDOW w as (partition by id order by ts)
|
||||
```
|
||||
Reference in New Issue
Block a user