如何使用ansible清除hadoopYarn缓存?

dfddblmv  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(551)

我想删除hadoop集群中的所有缓存文件夹。我已经写下了一些步骤,但是我不确定我应该如何继续写一个ansible脚本。总的来说,我理解ansible,但我有点搞不清楚我应该使用哪些模块来满足我的需求。
步骤:
停止Yarn服务。

$ sudo service hadoop-mapreduce-historyserver stop
$ sudo service hadoop-yarn-resourcemanager stop
$ sudo service hadoop-yarn-nodemanager stop

登录到所有节点并删除用户缓存目录的内容。例如:

for i in cat list_of_nodes_in_cluster; do ssh $i rm -rf
/prod/HDOP/data/13/daas.na/yarn/nm/usercache/*; done

验证所有节点中的所有用户缓存目录是否为空。
启动Yarn服务。

$ sudo service hadoop-mapreduce-historyserver start
$ sudo service hadoop-yarn-resourcemanager start
$ sudo service hadoop-yarn-nodemanager start

将消息“Yarn缓存已刷新”发送到我们的组id。

uqxowvwt

uqxowvwt1#

您可以使用此ansible playbook执行任务。
停止Yarn服务。

- name: service hadoop-mapreduce-historyserver stop
  service: 
    name: hadoop-mapreduce-historyserver
    state: stopped

- name: service hadoop-yarn-resourcemanager stop
  service: 
    name: hadoop-yarn-resourcemanager
    state: stopped

- name: service hadoop-yarn-nodemanager stop
  service: 
    name: hadoop-yarn-nodemanager
    state: stopped

登录到所有节点并删除用户缓存目录的内容。

- name: delete the content of the usercache directories
   file: path="{{ item }}" state=absent
   with_fileglob:
       - /prod/HDOP/data/13/daas.na/yarn/nm/usercache/*

验证所有节点中的所有用户缓存目录是否为空。

- name:  Verify all usercache directories are empty.
    shell: /usr/bin/ls ~/soniya/ | wc -l
    register: total_size

  - debug:
         msg: "Directory total size is {{ total_size.stdout_lines }}"
        when: total_size.rc == 0

启动Yarn服务。

- name: service hadoop-mapreduce-historyserver start
  service: 
    name: hadoop-mapreduce-historyserver
    state: started

- name: service hadoop-yarn-resourcemanager start
  service: 
    name: hadoop-yarn-resourcemanager
    state: started

- name: service hadoop-yarn-nodemanager start
  service: 
    name: hadoop-yarn-nodemanager
    state: started

将“Yarn缓存已刷新”消息发送到我们的组id
您可以使用邮件模块进行此操作。

相关问题