Elasticsearch @jolestar
目录 Elasticsearch 简介 Elasticsearch 核心概念及架构 Elasticsearch 集群搭建与配置 Elasticsearch 集群演示 Elasticsearch 全文检索 Elasticsearch 文档数据库 Elasticsearch 分析引擎 ELK 使用场景演示
Elasticsearch 简介 Elasticsearch is a distributed, open source search and analytics engine, designed for horizontal scalability, reliability, and easy management. It combines the speed of search with the power of analytics via a sophisticated, developer-friendly query language covering structured, unstructured, and time-series data.
Elasticsearch 简介 - 使用场景 站内搜索 ( 全文索引 ) 文档数据库 (vs mongodb) 日志以及时间序列数据 (ELK)
Elasticsearch 核心概念 集群 (Cluster) 节点 (Node) 索引 (Index) 主分片 (Primary shard) 副本分片 (Replica shard) 类型 (Type) Mapping 文档 (Document) 字段 (Field) 分配 (Allocation)
Elasticsearch 架构 - 分布式 状态共享 服务发现 选主 弹性 (Elastic) 新增节点 删除节点
Elasticsearch 架构 - 服务发现以及选主 节点启动后先 ping( 这里的 ping 是 Elasticsearch 的一个 RPC 命令 ) Ping 的 response 会包含该节点的基本信息以及该节点认为的 master 节点 选举开始, 先从各节点认为的 master 中选, 规则很简单, 按照 id 的字典序排序, 取第一个 如果各节点都没有认为的 master, 则从所有节点中选择, 规则同上 这里有个限制条件就是 discovery.zen.minimum_master_nodes, 如果节点数达不到最小值的限制, 则循环上述过程, 直到节点数足够可以开始选举 最后选举结果是肯定能选举出一个 master, 如果只有一个 local 节点那就选出的是自己 如果当前节点是 master, 则开始等待节点数达到 minimum_master_nodes, 然后提供服务 如果当前节点不是 master, 则尝试加入 master
Elasticsearch 架构 - 分片以及副本
Elasticsearch 架构 - 恢复与容灾 集群中的某个节点丢失网络连接 master 提升该节点上的所有主分片的在其他节点上的副本为主分片 cluster 集群状态变为 yellow, 因为副本数不够 等待一个超时设置的时间, 如果丢失节点回来就可以立即恢复 ( 默认为 1 分钟, 通过 index.unassigned.node_left.delayed_timeout 设置 ) 如果该分片已经有写入, 则通过 translog 进行增量同步数据 否则将副本分配给其他节点, 开始同步数据
Elasticsearch 架构 - 系统架构 Guice Netty Lucene ClusterState
Elasticsearch 集群搭建 java 下载 https://download.elastic.co/elasticsearch/release/org/ elasticsearch/distribution/tar/elasticsearch/2.3.5/ elasticsearch-2.3.5.tar.gz bin/elasticsearch config/elasitcsearch.yaml
Elasticsearch 集群搭建 -config bootstrap.mlockall: true cluster.name: elasticsearch discovery.zen.minimum_master_nodes: 2 discovery.zen.ping.timeout: 5s discovery.zen.ping.unicast.hosts: ["192.168.229.11", "192.168.229.5"] gateway.recover_after_nodes: 2 http.port: 9200 network.host: 0.0.0.0 node.name: node-1 path.data: /data/elasticsearch/data path.logs: /data/elasticsearch/logs script.file: false script.indexed: sandbox script.inline: sandbox script.mapping: false script.update: false http.cors.enabled: true http.cors.allow-origin: "*" index.number_of_shards: 1 index.number_of_replicas: 0
Elasticsearch 集群搭建 -docker docker pull elasticsearch:2.3.5 docker run --name elasticsearch -p 9200:9200 -p 9300:9300 -d elasticsearch:2.3.5 elasticsearch
Elasticsearch 集群演示
Elasticsearch 全文检索 Lucene Mapper-attachments Analyzer
Elasticsearch 全文检索 -lucene http://www.slideshare.net/gamgoster/architecture-and-implementation-of-apache-lucene-13105167
Elasticsearch 全文检索 -lucene
Elasticsearch 全文检索 -lucene
Elasticsearch 文档数据库 Lucene store field Translog Dynamic-mapping 以及 schema-free QueryDSL
Elasticsearch 文档数据库 -mapping PUT my_index { "mappings": { "user": { "_all": { "enabled": false, "properties": { "title": { "type": "string", "name": { "type": "string", "age": { "type": "integer", "blogpost": { "properties": { "title": { "type": "string", "body": { "type": "string", "user_id": { "type": "string", "index": "not_analyzed", "created": { "type": "date", "format": "strict_date_optional_time epoch_millis"
Elasticsearch 文档数据库 -datatype
Elasticsearch 文档数据库 -QueryDSL SELECT document FROM products WHERE price = 20 { "term" : { "price" : 20
Elasticsearch 文档数据库 -QueryDSL SELECT product FROM products WHERE (price = 20 OR productid = "XHDK-A-1293-#fJ3") AND (price!= 30) { { "bool" : { "must" : [], "should" : [], "must_not" : [], "filter": [] "bool" : { "should" : [ { "term" : {"price" : 20, { "term" : {"productid" : "XHDK- A-1293-#fJ3" ], "must_not" : { "term" : {"price" : 30
Elasticsearch 文档数据库 -QueryDSL SELECT document FROM products WHERE price BETWEEN 20 AND 40 "range" : { "price" : { "gt" : 20, "lt" : 40
Elasticsearch 分析引擎 Aggregations Kibana
Elasticsearch 分析引擎 -Aggregations Bucketing Metric Pipeline
Elasticsearch 分析引擎 -Aggregations "aggs" : { "<aggregation_name>" : { "<aggregation_type>" : { <aggregation_body> [,"meta" : { [<meta_data_body>] ]? [,"aggregations" : { [<sub_aggregation>]+ ]? [,"<aggregation_name_2>" : {... ]*
Elasticsearch 分析引擎 -Aggregations { { "aggs" : { "avg_grade" : { "avg" : { "field" : "grade" "aggs" : { "articles_over_time" : { "date_histogram" : { "field" : "date", "interval" : "month"
Elasticsearch 分析引擎 -Serial Differencing Aggregation
Elasticsearch 分析引擎 -Aggregations { "aggs": { "my_date_histo": { "date_histogram": { "field": "timestamp", "interval": "day", "aggs": { "the_sum": { "sum": { "field": "lemmings", "thirtieth_difference": { "serial_diff": { "buckets_path": "the_sum", "lag" : 30
Elasticsearch 分析引擎 -Aggregations
ELK 演示
Thank you. @jolestar