环境要求:

1. elasticsearch curator(与zookeeper curator不一样)

2. crontab

安装curator

安装pip

[root@es1 ~]# wget https://bootstrap.pypa.io/get-pip.py--2016-12-12 11:18:34--  https://bootstrap.pypa.io/get-pip.pyResolving bootstrap.pypa.io... 151.101.16.175Connecting to bootstrap.pypa.io|151.101.16.175|:443... connected.HTTP request sent, awaiting response... 200 OKLength: 1595408 (1.5M) [text/x-python]Saving to: et-pip.py100%[====================================================================================================================>] 1,595,408   9.40K/s   in 2m 3s   [root@es1 ~]# python get-pip.py DEPRECATION: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of pip will drop support for Python 2.6Collecting pip/tmp/tmpONfJtN/pip.zip/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning./tmp/tmpONfJtN/pip.zip/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.  Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)    100% || 1.3MB 58kB/s Collecting setuptools  Downloading setuptools-31.0.0-py2.py3-none-any.whl (477kB)    100% || 481kB 55kB/s Collecting wheel  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)    100% || 71kB 35kB/s Collecting argparse; python_version == "2.6" (from wheel)  Downloading argparse-1.4.0-py2.py3-none-any.whlInstalling collected packages: pip, setuptools, argparse, wheelSuccessfully installed argparse-1.4.0 pip-9.0.1 setuptools-31.0.0 wheel-0.29.0/tmp/tmpONfJtN/pip.zip/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.

安装curator

[root@es1 ~]# pip install elasticsearch-curatorDEPRECATION: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of pip will drop support for Python 2.6Collecting elasticsearch-curator/usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.  SNIMissingWarning/usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.  InsecurePlatformWarning  Downloading elasticsearch-curator-4.2.4.tar.gz (134kB)    100% || 143kB 54kB/s Collecting elasticsearch<3.0.0,>=2.4.0 (from elasticsearch-curator)  Downloading elasticsearch-2.4.0-py2.py3-none-any.whl (54kB)    100% || 61kB 64kB/s Collecting click>=6.0 (from elasticsearch-curator)  Downloading click-6.6-py2.py3-none-any.whl (71kB)    100% || 71kB 43kB/s Collecting pyyaml>=3.10 (from elasticsearch-curator)  Downloading PyYAML-3.12.tar.gz (253kB)    100% || 256kB 55kB/s Collecting voluptuous>=0.9.3 (from elasticsearch-curator)  Downloading voluptuous-0.9.3.tar.gzCollecting certifi>=2016.09.26 (from elasticsearch-curator)  Downloading certifi-2016.9.26-py2.py3-none-any.whl (377kB)    100% || 378kB 58kB/s Collecting urllib3<2.0,>=1.8 (from elasticsearch<3.0.0,>=2.4.0->elasticsearch-curator)  Downloading urllib3-1.19.1-py2.py3-none-any.whl (104kB)    100% || 112kB 48kB/s Requirement already satisfied: setuptools>=0.6b1 in /usr/lib/python2.6/site-packages (from voluptuous>=0.9.3->elasticsearch-curator)Building wheels for collected packages: elasticsearch-curator, pyyaml, voluptuous  Running setup.py bdist_wheel for elasticsearch-curator ... done  Stored in directory: /root/.cache/pip/wheels/33/46/ac/5e17bb2d61b2139011742f37ca0a2d7fde039cca3d51b574f8  Running setup.py bdist_wheel for pyyaml ... done  Stored in directory: /root/.cache/pip/wheels/2c/f7/79/13f3a12cd723892437c0cfbde1230ab4d82947ff7b3839a4fc  Running setup.py bdist_wheel for voluptuous ... done  Stored in directory: /root/.cache/pip/wheels/4b/cc/a6/c1683326047075d71cdd74c966300efbfeda143fdf6c5f7587Successfully built elasticsearch-curator pyyaml voluptuousInstalling collected packages: urllib3, elasticsearch, click, pyyaml, voluptuous, certifi, elasticsearch-curatorSuccessfully installed certifi-2016.9.26 click-6.6 elasticsearch-2.4.0 elasticsearch-curator-4.2.4 pyyaml-3.12 urllib3-1.19.1 voluptuous-0.9.3

检查curator

[root@es1 ~]# curator --helpUsage: curator [OPTIONS] ACTION_FILE  Curator for Elasticsearch indices.  See http://elastic.co/guide/en/elasticsearch/client/curator/currentOptions:  --config PATH  Path to configuration file. Default: ~/.curator/curator.yml  --dry-run      Do not perform any changes.  --version      Show the version and exit.  --help         Show this message and exit.[root@es1 ~]# curator_cli --helpUsage: curator_cli [OPTIONS] COMMAND [ARGS]...Options:  --config PATH       Path to configuration file. Default:                      ~/.curator/curator.yml  --host TEXT         Elasticsearch host.  --url_prefix TEXT   Elasticsearch http url prefix.  --port TEXT         Elasticsearch port.  --use_ssl           Connect to Elasticsearch through SSL.  --certificate TEXT  Path to certificate to use for SSL validation.  --client-cert TEXT  Path to file containing SSL certificate for client auth.  --client-key TEXT   Path to file containing SSL key for client auth.  --ssl-no-validate   Do not validate SSL certificate  --http_auth TEXT    Use Basic Authentication ex: user:pass  --timeout INTEGER   Connection timeout in seconds.  --master-only       Only operate on elected master node.  --dry-run           Do not perform any changes.  --loglevel TEXT     Log level  --logfile TEXT      log file  --logformat TEXT    Log output format [default|logstash|json].  --version           Show the version and exit.  --help              Show this message and exit.Commands:  allocation        Shard Routing Allocation  close             Close indices  delete_indices    Delete indices  delete_snapshots  Delete snapshots  forcemerge        forceMerge index/shard segments  open              Open indices  replicas          Change replica count  show_indices      Show indices  show_snapshots    Show snapshots  snapshot          Snapshot indices

配置curator

配置客户端连接

# curator_conf.ymlclient:    hosts: ["172.16.1.42:9200"]    url_prefix:     use_ssl: False    certificate:     client_cert:     client_key:     ssl_no_validate: False    http_auth:     timeout: 30    master_only: Falselogging:    loglevel: INFO    logfile: /data/apps/curator/curator.log     logformat: default    blacklist: ['elasticsearch','urllib3']

配置执行动作

# curator_actions.yml---# Remember, leave a key empty if there is no value.  None will be a string,# not a Python "NoneType"## Also remember that all examples have 'disable_action' set to True.  If you# want to use this action as a template, be sure to set this to False after# copying it.actions:  1:    action: delete_indices    description: >-      Delete indices older than 3 days (based on index name), for baihe_ha-      prefixed indices. Ignore the error if the filter does not result in an      actionable list of indices (ignore_empty_list) and exit cleanly.    options:      ignore_empty_list: True      timeout_override:      continue_if_exception: False      disable_action: False     filters:    - filtertype: pattern      kind: prefix      value: bh_ha-      exclude:    - filtertype: age      source: name      direction: older      timestring: '%Y.%m.%d'      unit: days      unit_count: 3      exclude:  2:    action: delete_indices    description: >-      Delete indices older than 7 days (based on index name), for logstash-      prefixed indices. Ignore the error if the filter does not result in an      actionable list of indices (ignore_empty_list) and exit cleanly.    options:      ignore_empty_list: True      timeout_override:      continue_if_exception: False      disable_action: False    filters:    - filtertype: pattern      kind: prefix      value: logstash-      exclude:    - filtertype: age      source: name      direction: older      timestring: '%Y.%m.%d'      unit: days      unit_count: 7      exclude:

测试curator脚本

curator --config ./curator_conf.yml ./curator_actions.yml

配置crontab

编写执行脚本

#!/bin/sh/usr/bin/curator --config /data/apps/curator/curator_conf.yml /data/apps/curator/curator_actions.yml#curator_start.sh

配置执行规则

crontab -e：打开了vi，输入：30 5 * * * /data/apps/curator/curator-start.sh，之后保存退出vicrontab -l：查看所有的root用户的定时任务 附：crontab的文件格式分 时 日 月 星期 要运行的命令第1列分钟1～59第2列小时1～23（0表示子夜）第3列日1～31第4列月1～12第5列星期0～7（0和7表示星期天）第6列要运行的命令

troubleshooting

[root@S3-202 ~]# curator --helpTraceback (most recent call last):  File "/usr/bin/curator", line 5, in 
    
         from pkg_resources import load_entry_point  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 3007, in 
     
          working_set.require(__requires__)  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 728, in require    needed = self.resolve(parse_requirements(requirements))  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 631, in resolve    requirements.extend(dist.requires(req.extras)[::-1])  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2489, in requires    dm = self._dep_map  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2700, in _dep_map    self.__dep_map = self._compute_dependencies()  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2717, in _compute_dependencies    from _markerlib import compile as compile_markerImportError: No module named _markerlib这并不是需要安装markerlib,只要升级一下setuptools即可,方法如下:wget https://pypi.python.org/packages/source/s/setuptools/setuptools-19.2.zipunzip setuptools-19.2.zipcd setuptools-19.2python setup.py install[root@es1 setuptools-19.2]# pip install elasticsearch-curatorDEPRECATION: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of pip will drop support for Python 2.6Requirement already satisfied: elasticsearch-curator in /usr/lib/python2.6/site-packagesRequirement already satisfied: certifi>=2016.09.26 in /usr/lib/python2.6/site-packages (from elasticsearch-curator)Requirement already satisfied: pyyaml>=3.10 in /usr/lib64/python2.6/site-packages (from elasticsearch-curator)Requirement already satisfied: voluptuous>=0.9.3 in /usr/lib/python2.6/site-packages (from elasticsearch-curator)Requirement already satisfied: click>=6.0 in /usr/lib/python2.6/site-packages (from elasticsearch-curator)Requirement already satisfied: elasticsearch<3.0.0,>=2.4.0 in /usr/lib/python2.6/site-packages (from elasticsearch-curator)Requirement already satisfied: setuptools>=0.6b1 in /usr/lib/python2.6/site-packages/setuptools-19.2-py2.6.egg (from voluptuous>=0.9.3->elasticsearch-curator)Requirement already satisfied: urllib3<2.0,>=1.8 in /usr/lib/python2.6/site-packages (from elasticsearch<3.0.0,>=2.4.0->elasticsearch-curator)
     
    

参考:

http://www.cnblogs.com/java-zhao/p/5900590.html

https://www.elastic.co/guide/en/elasticsearch/client/curator/current/about.html