解决方法如下：
1） 我们通过pyhive游标逐块检索数据。
2） 预处理：编码/解码。
3） 附加到最终Dataframe。

# cursor to the database.

cursor = hive.Connection(host=HOST, port=PORT, username=USERNAME).cursor()

# execute the query on the database side.

cursor.execute("SELECT id, message FROM table")

# result dataframe, empty for now.

df = pd.DataFrame(columns=['id', 'message'])

while True:
    # fetch 10k rows (as tuples).
    rows = cursor.fetchmany(10000)

    # if no more rows to retrieve, we stop.
    if not rows:
        break

    # Preprocessing: do encoding/decoding here
    rows = [(id, message.decode('utf-8', 'ignore')) for id, message in rows]

    # put result in a temporary dataframe
    df_tmp = pd.DataFrame(rows, columns=['id', 'message'])

    # merge the temporary dataframe to the original df
    df = pd.concat([df, df_tmp])

df = ...