安装

基于 MacOS 安装

1
2
3
brew install elasticsearch // port 9200
brew install kibana // port 5601
## 注：注意版本兼容性；默认安装最新版

ES 命令行操作

查询索引：curl http://10.0.80.167:9200/_cat/indices

查询所有模板名称：curl -XGET http://10.0.80.167:9200/_cat/templates/

查询模板内容：curl -XGET http://10.0.80.167:9200/_template/dreamhomes

查询数据：curl -XGET http://10.0.80.167:9200/ccb-trans-2021-03-08/_search\?pretty

删除索引：curl -XDELETE http://10.0.80.167:9200/index1,index2

命令行聚合：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
curl -XPOST http://10.0.80.167:9200/index/_search\?pretty -H'Content-Type: application/json' -d' 
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "st": {
              "gt": 1615135500000,
              "lt": 1615135620000
            }
          }
        },
        {
          "term": {
            "tc": "A000"
          }
        }
      ]
    }
  },
  "aggs": {
    "time": {
      "histogram": {
        "field": "st",
        "interval": 60000
      },
      "aggs": {
        "ret_code": {
          "terms": {
            "field": "ret.keyword"
          },
          "aggs": {
            "avg_latency": {
              "avg": {
                "field": "cost"
              }
            },
            "bussc_cnt": {
              "sum": {
                "field": "bussc"
              }
           },
            "syssc_cnt": {
              "sum": {
                "field": "syssc"
              }
            }
          }
        }
      }
    }
  }
}'

设置索引字段格式模板

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// ES 6.8.9 设置索引模板
PUT /_template/dreamhomes
{
  "index_patterns": [
    "ccb*"
  ],
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
    "_doc": {
      "dynamic_templates": [
        {
          "strings_as_keywords": {
            "match_mapping_type": "string",
            "mapping": {
              "type": "keyword"
            }
          }
        }
      ],
      "properties": {
        "@timestamp": {
          "type": "date",
          "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'||epoch_millis"
        }
      }
    }
  }
}

📢 注意：ES 聚合字段默认返回数量为 10，如果查询超过 10 个聚合字段时需要指定数量大小，给出示例如下

1
2
3
4
5
6
7
8
9
10
11
12
GET https://10.0.90.74:9200/index/_search
{
    "size": 0,
    "aggs": {
        "aggs": {
            "terms": {
                "field": "type",
                "size": 10000
            }
        }
    }
}

Python API

  • 创建连接

    1
    2
    3
    4
    5
    6
    ip_address = "http://10.0.80.167:10018/"

    es = Elasticsearch(hosts=ip_address) # http_auth=('user','pws')

    # 查看所有索引名
    es.indices.get_alias()

  • 查询

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    // 选择关键属性
    csv_header = [ "timestamp", "transstatus", "traceNo",  "return_type", "hostip", "returncode", "transcode"]
    body = {
        "query":{
            "match_all":{}
        },
        "_source": {
            "includes": csv_header,
            "excludes": []
        }
    }

    es_index = "ecbp-2020-03-31"
    es_type = "_doc"

    query = es.search(index=es_index, doc_type=es_type, body=body)

Kibana 基本操作

修改配置连接远程es ：config/kibana.yml

1
elasticsearch.hosts: "http://10.0.80.167:9200" #远程es地址