网络游戏 2017-04-07
使用C#代码实现对ElastiSearch的编程查询,是十分方便的,通常情况下,开发者采用官方提供的NEST客户端程序,通过封装的方法向ElasticSearch引擎发送查询请求,搜索数据,最终获取返回的查询结果,实现预定的业务需求。在内部,NEST客户端通过格式化的数据结构,把C#代码转换成HTTP 请求(Request),减轻了用户直接编写Qeury DSL的麻烦。当然,用户也可以直接把Query DSL封装成HTTP请求,发送到ElasticSearch引擎;对开发者来说,不仅需要熟悉Query DSL的语法,而且需要手动编写代码,处理引擎返回的JSON结构化的数据集,采用这种方式的优点是不受限于NEST客户端程序,能够最大化使用ElasticSearch查询的各种参数,书写自由。
本文简单介绍使用C#代码对ElasticSearch进行编程查询的流程,具体的细节,请参考官方文档。
一,编程流程
1,创建客户端
在搜索文档之前,首先要连接到ElasticSearch引擎,创建客户端对象
using Nest; var node = new Uri("http://myserver:9200"); var settings = new ConnectionSettings(node).DefaultIndex("default"); var client = new ElasticClient(settings);
2,创建查询请求
连接到引擎之后,创建搜索请求(SearchRequest),用于封装查询类型和查询条件
SearchRequest sr = new SearchRequest("meetup", "events");
3,指定查询类型和查询条件
为搜索请求指定查询类型,可以是词条搜索,或全文搜索
TermQuery tq = new TermQuery(); tq.Field = "eventname"; tq.Value = "azure"; sr.Query = tq;
4,调整查询结果
为搜索请求设置参数,排序,分页,和选择返回的字段等,在选择查询结果返回的字段时,推荐在查询请求(SearchRequest)中使用Source Filter。
在查询请求中,通过类RequestSearch的数组字段StoredFileds,把已存储字段添加到该数组中,ElasticSearch引擎只返回特定的字段,而不是文档的所有字段。在索引映射中,已存储字段的store属性为true,StoredFileds数组只能选择已存储字段(stored field)。
//windows sr.From = 0; sr.Size = 100; //sort ISort sort = new SortField { Field = "eventid", Order = SortOrder.Ascending }; sr.Sort = new List<ISort>(); sr.Sort.Add(sort); //source filter sr.Source = new SourceFilter() { Includes = new string[] { "eventid", "eventname" }, Excludes = new string[] { "roginalid", "description" } };
5,执行查询请求
最后,客户端执行搜索请求,获取搜索结果,并将查询结果中的文档集转换成列表
var result = client.Search<MeetupEvents>(sr); return result.Documents.ToList<MeetupEvents>();
二,示例代码,使用Nest客户端搜索文档
在该示例代码中,本文简单列举词条查询,匹配查询,布尔查询和正则表达式查询的示例代码。
1,词条查询
public List<MeetupEvents>GetResult_TermQuery( ) { //create term query TermQuery tq = new TermQuery(); tq.Field = "eventname"; tq.Value = "azure"; //create search request SearchRequest sr = new SearchRequest("meetup", "events"); sr.Query = tq; //windows sr.From = 0; sr.Size = 100; //sort ISort sort = new SortField { Field = "eventid", Order = SortOrder.Ascending }; sr.Sort = new List<ISort>(); sr.Sort.Add(sort); //source filter sr.Source = new SourceFilter() { Includes = new string[] { "eventid", "eventname" }, Excludes = new string[] { "roginalid", "description" } }; var result = client.Search<MeetupEvents>(sr); return result.Documents.ToList<MeetupEvents>(); }
2,匹配查询
public List<MeetupEvents> GetResult_MatchQuery() { SearchRequest sr = new SearchRequest("meetup", "events"); MatchQuery mq = new MatchQuery(); mq.Field = new Field("eventname"); mq.Query = "azure cloud"; mq.MinimumShouldMatch = 2; mq.Operator = Operator.Or; sr.Query = mq; sr.From = 0; sr.Size = 100; sr.Sort = new List<ISort>(); sr.Sort.Add(new SortField { Field = "eventid", Order = SortOrder.Ascending }); ISearchResponse<MeetupEvents> result = client.Search<MeetupEvents>(sr); return result.Documents.ToList<MeetupEvents>(); }
3,正则表达式查询
public List<MeetupEvents>GetResult_RegexpQuery() { SearchRequest sr = new SearchRequest(); RegexpQuery rq = new RegexpQuery(); rq.Field = "description"; rq.Value = "azu.*"; rq.MaximumDeterminizedStates = 20000; sr.Query = rq; var result = client.Search<MeetupEvents>(sr); return result.Documents.ToList<MeetupEvents>(); }
4,布尔查询
public List<MeetupEvents>GetResult_BoolQuery() { SearchRequest sr = new SearchRequest("meetup", "events"); BoolQuery bq = new BoolQuery(); bq.Filter = new QueryContainer[] { new MatchQuery() { Field="eventname", Query="azure cloud", Operator=Operator.Or, MinimumShouldMatch=1 }, new MatchQuery() { Field ="eventname", Query="aws google", Operator=Operator.Or, MinimumShouldMatch=1 } }; bq.Should = new QueryContainer[] { new TermQuery() { Field="description", Value="azure" }, new TermQuery() { Field="description", Value="cloud" } }; bq.MinimumShouldMatch = 1; sr.Query = bq; var result = client.Search<MeetupEvents>(sr); return result.Documents.ToList<MeetupEvents>(); }
三,把Query DSL封装成HTTP Request
向ElasticSearch引擎发送Http请求,在http请求中指定查询的类型和查询条件,引擎在收到请求后执行搜索,查询结果以HTTP 响应(Response)返回,开发者需要从Response返回的JSON结构字符串中解析搜索结果。
1,封装类库
以下HTTP网络编程代码,是我们项目组一姐Amy的作品,谢谢Amy的分享,代码可以进一步封装,在此文中,仅仅作为演示:
namespace ElasticSearchNet { class ESRequest { string es_host; string es_port; string es_index; string es_type; private string url; public ESRequest(string host,string index,string type,string port="9200") { es_host = host; es_port = port; es_index = index; es_type = type; string requst_cache = "request_cache=true"; url = string.Format("http://{0}:{1}/{2}/{3}/_search?{4}", es_host, es_port, es_index, es_type,requst_cache); } public string ExecuteQeury(string json_query) { HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url); request.ContentType = "aplication/json"; request.Method = "POST"; request.Timeout = 1000 * 60; using (var sw = new StreamWriter(request.GetRequestStream())) { sw.Write(json_query); sw.Flush(); sw.Close(); } var response = (HttpWebResponse)request.GetResponse(); using (var sr = new StreamReader(response.GetResponseStream())) { return sr.ReadToEnd(); } } } }View Code
2,执行查询
查询的结果是JSON结构的字符串,通常使用JObject和JToken类处理。
ESRequest es = new ElasticSearchNet.ESRequest("cia-sh-svr-sis3", "meetup", "events"); string json_query = @" { ""query"":{ ""match"":{ ""eventname"":""azure"" } } } "; string strJsonResult=es.ExecuteQeury(json_query);
解析JSON的常用类库是:
参考文档:
Elasticsearch.Net and NEST: the .NET clients [5.x] » Search
Elasticsearch.Net and NEST: the .NET clients [5.x] » Query DSL
另外一部分,则需要先做聚类、分类处理,将聚合出的分类结果存入ES集群的聚类索引中。数据处理层的聚合结果存入ES中的指定索引,同时将每个聚合主题相关的数据存入每个document下面的某个field下。