Elasticsearch搜索服务学习之七——分析雇员信息

高新技术,ElasticSearch

2017-06-05

178

0

目录


Elasticsearch 有一个功能叫聚合(aggregations),允许我们基于数据生成一些精细的分析结果。聚合与 SQL 中的 GROUP BY 类似但更强大。

举个例子,挖掘出雇员中最受欢迎的兴趣爱好:

GET /megacorp/employee/_search
{
  "aggs": {
    "all_interests": {
      "terms": { "field": "interests" }
    }
  }
}

执行情况如下:

belonk@DESKTOP-7TSK7UJ MINGW64 ~
$ curl -XGET 'localhost:9200/megacorp/employee/_search?pretty' -H 'Content-Type: application/json' -d'
> {
>   "aggs": {
>     "all_interests": {
>       "terms": { "field": "interests" }
>     }
>   }
> }
> '
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1751  100  1661  100    90  21025   1139 --:--:-- --:--:-- --:--:-- 21025{
  "took" : 68,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 1.0,
    "hits" : [
      …… // 此处省略命中结果
    ]
  },
  "aggregations" : {
    "all_interests" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "music",
          "doc_count" : 2
        },
        {
          "key" : "forestry1",
          "doc_count" : 1
        },
        {
          "key" : "sports",
          "doc_count" : 1
        }
      ]
    }
  }
}

可以看到,两位员工对音乐感兴趣,一位对林地感兴趣,一位对运动感兴趣。这些聚合并非预先统计,而是从匹配当前查询的文档中即时生成。如果想知道叫 Smith 的雇员中最受欢迎的兴趣爱好,可以直接添加适当的查询来组合查询:

GET /megacorp/employee/_search
{
  "query": {
    "match": {
      "last_name": "smith"
    }
  },
  "aggs": {
    "all_interests": {
      "terms": {
        "field": "interests"
      }
    }
  }
}

all_interests 聚合已经变为只包含匹配查询的文档:

belonk@DESKTOP-7TSK7UJ MINGW64 ~
$ curl -XGET 'localhost:9200/megacorp/employee/_search?pretty' -H 'Content-Type: application/json' -d'
> {
>   "query": {
>     "match": {
>       "last_name": "smith"
>     }
>   },
>   "aggs": {
>     "all_interests": {
>       "terms": {
>         "field": "interests"
>       }
>     }
>   }
> }
> '
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1417  100  1247  100   170  15987   2179 --:--:-- --:--:-- --:--:-- 15987{
  "took" : 56,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
   ……
    ]
  },
  "aggregations" : {
    "all_interests" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "music",
          "doc_count" : 2
        },
        {
          "key" : "sports",
          "doc_count" : 1
        }
      ]
    }
  }
}

聚合还支持分级汇总 。比如,查询特定兴趣爱好员工的平均年龄:

GET /megacorp/employee/_search
{
    "aggs" : {
        "all_interests" : {
            "terms" : { "field" : "interests" },
            "aggs" : {
                "avg_age" : {
                    "avg" : { "field" : "age" }
                }
            }
        }
    }
}

得到的聚合结果有点儿复杂,但理解起来还是很简单的:

belonk@DESKTOP-7TSK7UJ MINGW64 ~
$ curl -XGET 'localhost:9200/megacorp/employee/_search?pretty' -H 'Content-Type: application/json' -d'
> {
>     "aggs" : {
>         "all_interests" : {
>             "terms" : { "field" : "interests" },
>             "aggs" : {
>                 "avg_age" : {
>                     "avg" : { "field" : "age" }
>                 }
>             }
>         }
>     }
> }
> '
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2099  100  1853  100   246  59774   7935 --:--:-- --:--:-- --:--:-- 59774{
  "took" : 23,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 1.0,
    "hits" : [
      ……
    ]
  },
  "aggregations" : {
    "all_interests" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "music",
          "doc_count" : 2,
          "avg_age" : {
            "value" : 28.5
          }
        },
        {
          "key" : "forestry1",
          "doc_count" : 1,
          "avg_age" : {
            "value" : 35.0
          }
        },
        {
          "key" : "sports",
          "doc_count" : 1,
          "avg_age" : {
            "value" : 25.0
          }
        }
      ]
    }
  }
}

输出基本是第一次聚合的加强版。依然有一个兴趣及数量的列表,只不过每个兴趣都有了一个附加的 avg_age 属性,代表有这个兴趣爱好的所有员工的平均年龄。

即使现在不太理解这些语法也没有关系,依然很容易了解到复杂聚合及分组通过 Elasticsearch 特性实现得很完美。可提取的数据类型毫无限制。

在测试这部分代码时,会遇到一个异常信息:

"Fielddata is disabled on text fields by default. Set fielddata=true on [interests] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.

我们将在下一篇分析下该异常。


前一篇:Elasticsearch搜索引擎学习之六——索引雇员文档
后一篇:Elasticsearch错误信息:Fielddata is disabled on text fields by default

belonk

轻轻地我走了,正如我轻轻地来,我挥一挥衣袖,不带走一片云彩