Elasticsearch错误信息:Fielddata is disabled on text fields by default

高新技术,ElasticSearch

2017-06-05

544

0

目录


上边文章我们简单尝试了使用聚合(aggregations)来分析统计雇员的兴趣爱好的人数、平均年龄等,在测试时,出现了一个400的错误信息:

Fielddata is disabled on text fields by default. Set fielddata=true on [your_field_name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.

原因

官方的关于fielddata的文档如下https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html

提到了该问题产生的原因:

Fielddata can consume a lot of heap space, especially when loading high cardinality text fields. Once fielddata has been loaded into the heap, it remains there for the lifetime of the segment. Also, loading fielddata is an expensive process which can cause users to experience latency hits. This is why fielddata is disabled by default.

If you try to sort, aggregate, or access values from a script on a text field, you will see this exception:

Fielddata is disabled on text fields by default. Set fielddata=true on [your_field_name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.

 大概的意思是说:

Fielddata会耗费大量的堆内存,尤其是在加载大文件时。一旦fielddata被加载到内存中,他会在segment的生命周期一直存在。而且,搜索时处理fielddata还会导致速度下降甚至严重延迟,所以默认fielddata被禁用。

如果你试图对text字段进行排序、聚合或者访问值等操作,你将会遇到如下异常:

Fielddata is disabled on text fields by default. Set fielddata=true on [your_field_name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.

解决办法

由于我们在测试时添加的都是文本类型的数据,需要对文本字段开启fielddata,需要使用 PUT mapping API,执行如下命令设置fielddata=true:

PUT my_index/_mapping/my_type
{
  "properties": {
    "my_field": { 
      "type":     "text",
      "fielddata": true
    }
  }
}

其中my_field字段是你需要进行排序、聚合等操作的包含现有映射的字段。

使用CURL执行情况如下:

belonk@DESKTOP-7TSK7UJ MINGW64 ~
$ curl -XPUT 'localhost:9200/megacorp/_mapping/employee' -d '
{
  "properties": {
    "interests": {
      "type":     "text",
      "fielddata": true
    }
  }
}'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   123  100    21  100   102    338   1645 --:--:-- --:--:-- --:--:--  1645{"acknowledged":true}

 设置fielddata为true后,再次测试脚本,该问题解决。


前一篇:Elasticsearch搜索服务学习之七——分析雇员信息
后一篇:git从其他git服务器完整迁移到自建git服务器

belonk

轻轻地我走了,正如我轻轻地来,我挥一挥衣袖,不带走一片云彩