es中springboot的两种连接方式

使用客户端调用es服务的两种方式

有两种方式,一种 是9200端口或叫rest 接口，
另一种是用 节点的9300端口或叫 TransportClient

说明

es的官网中https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-api.html
We plan on deprecating the TransportClient in Elasticsearch 7.0 and removing it completely in 8.0. Instead, you should be using the Java High Level REST Client, which executes HTTP requests rather than serialized Java requests. The migration guidedescribes all the steps needed to migrate.
The Java High Level REST Client currently has support for the more commonly used APIs, but there are a lot more that still need to be added. You can help us prioritise by telling us which missing APIs you need for your application by adding a comment to this issue: Java high-level REST client completeness.
Any missing APIs can always be implemented today by using the low level Java REST Client with JSON request and response bodies.

所以 ，后面都 是采用9200端口的！！！

官网中会说到low-level REST client f High Level REST Client,这两个和 transportclient没关系!!!

aws 连elasticsearch 的 坑

项目 中是使用 transport来调用es的，但是在aws中，这种 方式是不支持的!!!,要用9200的方式
具体情况:
https://forums.aws.amazon.com/thread.jspa?messageID=683536

不过这个还好，官网也说了，之后高版本会放弃这个,所以早做改变也是OK的.
org.springframework.data.elasticsearch.core中的ElasticsearchOperations 是一个接口，可以多种 实现!

使用springframework.data的情况(Transport), 不建议使用，请查看下一章.

怎么连es，使用第一种 方式

这边是使用spring boot的方式

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

spring对es地址及相关配置.
spring.data.elasticsearch.cluster-nodes=xxxx:9300,xxx:9301
spring.data.elasticsearch.cluster-name=b_cluster

怎么使用

直接引用即可

聚合,

//聚合操作
TermsAggregationBuilder tbuild = AggregationBuilders.terms("sa_groupid").field("said");
AggregationBuilder sab = AggregationBuilders.stats("a_stat").field("player_count");
tbuild.subAggregation(sab);
tbuild.size(saidlist.size());
SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withQuery(boolQueryBuilder).withIndices("aaa_online_log").withTypes("doc").withSearchType(SearchType.DEFAULT)
        .addAggregation(tbuild)
        .build();

Aggregations aggregations = elasticsearchTemplate.query(searchQuery, new ResultsExtractor<Aggregations>() {
    @Override
    public Aggregations extract(SearchResponse response) {
        return response.getAggregations();
    }
});

//聚合的内容获取!
MultiBucketsAggregation targetAggregation = aggregations.get("sa_groupid");
Map<Integer, DAggOnlineStat> sa_stat = new HashMap<>();
targetAggregation.getBuckets().forEach(tmpbuck ->{
    DAggOnlineStat dAggOnlineStat = new DAggOnlineStat();
    dAggOnlineStat.setApp_id(((Long)tmpbuck.getKey()).intValue());
    dAggOnlineStat.setDoc_count((int) tmpbuck.getDocCount());

    Aggregation tmpa = tmpbuck.getAggregations().getAsMap().get("a_stat");
    Stats tmpstats = (Stats) tmpa;

    dAggOnlineStat.setAvg(tmpstats.getAvg());
    dAggOnlineStat.setMax(((Double)tmpstats.getMax()).intValue());
    dAggOnlineStat.setMin(((Double)tmpstats.getMin()).intValue());
    sa_stat.put(dAggOnlineStat.getApp_id(), dAggOnlineStat);
});
return sa_stat;

查询单表

        String indexName = String.format("a_ooo_price_%s_log", currency_flag);

        BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
        boolQueryBuilder.must(new TermQueryBuilder("appid", appid));

        //按时间从早到晚排序
        SortBuilder sortBuilder = new FieldSortBuilder("current_time").order(SortOrder.ASC);

        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQueryBuilder).withIndices(indexName).withTypes("doc").withSort(sortBuilder)
                .build();

        SearchHits searchHits = elasticsearchTemplate.query(searchQuery, new ResultsExtractor<SearchHits>() {
            @Override
            public SearchHits extract(SearchResponse response) {
                return response.getHits();
            }
        });

        List<sfoogamePriceLog> sfoogamePriceLogs = new ArrayList<>();
        for (SearchHit searchHit : searchHits) {
            Map<String, Object> hitMap = searchHit.getSource();
            sfoogamePriceLog sfoogamePriceLog = new sfoogamePriceLog();
            sfoogamePriceLog.setId(searchHit.getId());
            sfoogamePriceLog.setDiscount_percent((Integer) hitMap.get("discount_percent"));
            sfoogamePriceLog.setUpdated((Long) hitMap.get("updated"));
            sfoogamePriceLog.setCdest((Integer) hitMap.get("cdest"));
            sfoogamePriceLog.setInitial((Integer) hitMap.get("initial"));
            sfoogamePriceLog.setCurrent_time((Long) hitMap.get("current_time"));
            sfoogamePriceLog.setCdate((String) hitMap.get("cdate"));
            sfoogamePriceLog.setAppid((Integer) hitMap.get("appid"));
            sfoogamePriceLog.setCurrency_flag((String) hitMap.get("currency_flag"));
            sfoogamePriceLogs.add(sfoogamePriceLog);
        }
        return sfoogamePriceLogs;

查询percent

PercentilesAggregationBuilder tbuild = AggregationBuilders.percentiles("b_sta").field("score");

BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
boolQueryBuilder.must(new TermQueryBuilder("language", "english"));

SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withQuery(boolQueryBuilder).withIndices("sfoogame_detail").withTypes("doc").withSearchType(SearchType.DEFAULT)
        .addAggregation(tbuild)
        .build();

Aggregations aggregations = elasticsearchTemplate.query(searchQuery, new ResultsExtractor<Aggregations>() {
    @Override
    public Aggregations extract(SearchResponse response) {
        return response.getAggregations();
    }
});

InternalTDigestPercentiles tmp = aggregations.get("b_sta");
Double percentile_val = tmp.percentile(50);
return percentile_val;

怎么连jest与使用,即9200端口.

官网:
https://github.com/VanRoy/spring-data-jest

com.github.vanroy spring-boot-starter-data-jest 3.1.5.RELEASE

官网使用的是xml的配置

    <!-- ElasticSearch Jest Client -->
    <bean id="jestClient" factory-bean="jestClientFactory" factory-method="getObject" destroy-method="shutdownClient" />

    <bean id="jestClientConfigBuilder" class="io.searchbox.client.config.HttpClientConfig.Builder">
        <constructor-arg type="java.lang.String" value="http://localhost:9200"/>
    </bean>

    <bean id="jestClientConfig" factory-bean="jestClientConfigBuilder" factory-method="build"/>

    <bean id="jestClientFactory" class="io.searchbox.client.JestClientFactory">
        <property name="httpClientConfig" ref="jestClientConfig"/>
    </bean>

    <!-- Elasticsearch Jest Template -->
    <bean id="jestElasticsearchTemplate" class="com.github.vanroy.springdata.jest.JestElasticsearchTemplate">
        <constructor-arg ref="jestClient"/>
    </bean>

由于我们是使用springboot的，这里从上面翻译成springboot中的配置

    @Value("${project.es.serveruri}")
    public String serverUri ;

    @Bean
    public io.searchbox.client.config.HttpClientConfig.Builder jestClientConfigBuilder() {
        io.searchbox.client.config.HttpClientConfig.Builder builder = new io.searchbox.client.config.HttpClientConfig.Builder(serverUri);
        return builder;
    }
    @Bean
    public HttpClientConfig jestClientConfig(@Qualifier("jestClientConfigBuilder") io.searchbox.client.config.HttpClientConfig.Builder builder) {
        return builder.build();
    }

    @Bean
    public io.searchbox.client.JestClientFactory jestClientFactory(@Qualifier("jestClientConfig") HttpClientConfig httpClientConfig){
        io.searchbox.client.JestClientFactory factory = new io.searchbox.client.JestClientFactory();
        factory.setHttpClientConfig(httpClientConfig);
        return factory;
    }

    @Bean
    public JestClient jestClient(@Qualifier("jestClientFactory")JestClientFactory jestClientFactory) {
        return jestClientFactory.getObject();
    }


    @Bean
    public com.github.vanroy.springdata.jest.JestElasticsearchTemplate jestElasticsearchTemplate(@Qualifier("jestClient")JestClient jestClient){
        JestElasticsearchTemplate jestElasticsearchTemplate = new JestElasticsearchTemplate(jestClient);
        return jestElasticsearchTemplate;
    }

jest怎么使用?

@Autowired
private JestElasticsearchTemplate elasticsearchTemplate;

直接使用是OK的，不过我找了很久都 有发现有靠谱的文档，对于使用template的资料很少，只能自己摸索,

使用template查单表数据

        String indexName = "aaa_index"

        BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
        boolQueryBuilder.must(new TermQueryBuilder("aid", aid));

        //按时间从早到晚排序
        SortBuilder sortBuilder = new FieldSortBuilder("current_time").order(SortOrder.ASC);

        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQueryBuilder).withIndices(indexName).withTypes("doc").withSort(sortBuilder)
                .build();

        List<SearchResult.Hit<AAA, Void>> searchHits = elasticsearchTemplate.query(searchQuery, new JestResultsExtractor<List<SearchResult.Hit<AAA, Void>>>() {
            @Override
            public List<SearchResult.Hit<AAA, Void>> extract(SearchResult searchResult) {
                return  searchResult.getHits(AAA.class);
            }
        });

        List<AAA> aaas = new ArrayList<>();
        for (SearchResult.Hit<AAA, Void> searchHit : searchHits) {
            AAA aaa = searchHit.source;

        }

AAA.class是一个自定义的pojo类,用来装内容的.

查max

        BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
        TermsQueryBuilder queryBuilder1 = new TermsQueryBuilder("aid",String.valueOf(aid));

        boolQueryBuilder.must(queryBuilder1);

        MaxAggregationBuilder tbuild = AggregationBuilders.max("count_max").field("player_count");


        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQueryBuilder).withIndices("project_online_peak").withTypes("doc").withSearchType(SearchType.DEFAULT)
                .addAggregation(tbuild)
                .build();

        MetricAggregation aggregations = elasticsearchTemplate.query(searchQuery, new JestResultsExtractor<MetricAggregation>() {
            @Override
            public MetricAggregation extract(SearchResult searchResult) {
                return  searchResult.getAggregations();
            }
        });


        Double count_max_double = null;
        try {
//            count_max_double = ((InternalMax)aggregations.get("count_max")).value();
            count_max_double = aggregations.getMaxAggregation("count_max").getMax();
        } catch (Exception e) {
            log.error("count_max获取失败", e);
        }
        return  count_max_double.intValue();

查percentiles数据

PercentilesAggregationBuilder tbuild = AggregationBuilders.percentiles("b_sta").field("score");

BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
boolQueryBuilder.must(new TermQueryBuilder("language", "english"));

SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withQuery(boolQueryBuilder).withIndices("sfoogame_detail").withTypes("doc").withSearchType(SearchType.DEFAULT)
        .addAggregation(tbuild)
        .build();

MetricAggregation aggregations = elasticsearchTemplate.query(searchQuery, new JestResultsExtractor<MetricAggregation>() {
    @Override
    public MetricAggregation extract(SearchResult searchResult) {
        return  searchResult.getAggregations();
    }
});


//        InternalTDigestPercentiles tmp = aggregations.get("b_sta");
Map<String, Double> tmp = aggregations.getPercentilesAggregation("b_sta").getPercentiles();

Double targetVal = null;
for (String percentilestr : tmp.keySet()) {
    Double tdouble = new Double(percentilestr);
    if(75 == tdouble.intValue()){ // %50以上
        targetVal = tmp.get(percentilestr);
    }
}

两种的一些使用差异.

在有些方法，会出现unsupportedOpertationException.对这些方法，我们要区别去改代码…【竟然不是在接口层去封装而是改业务代码，真是诡异】,
调这些方法会抛错

java.lang.UnsupportedOperationException
    at com.github.vanroy.springdata.jest.JestElasticsearchTemplate.query(JestElasticsearchTemplate.java:413)

现在来看，换个方法即可

参考:

https://help.aliyun.com/document_detail/69194.html?spm=a2c4g.11186623.6.552.45263075k2X9j5