Search
// lets use a slightly different model class this time
data class Thing(val name: String)
Lets index some documents to look for …
// force ES to commit everything to disk so search works right away
.bulk(refreshPolicy = WriteRequest.RefreshPolicy.IMMEDIATE) {
repo("1", Thing("The quick brown fox"))
index("2", Thing("The quick brown emu"))
index("3", Thing("The quick brown gnu"))
index("4", Thing("Another thing"))
index5.rangeTo(100).forEach {
("$it", Thing("Another thing: $it"))
index}
}
.refresh() repo
Searching and working with results
// a SearchRequest is created and passed into the block
val results = repo.search {
// we can use Kotlin's string templating
val text = "brown"
{
configure = match("name", text)
query }
}
("Found ${results.totalHits}")
println
// we can get the deserialized thing from the search response
.mappedHits.forEach {
results// kotlin sequences are lazy; nothing is deserialized unless you use it
(it)
println}
// we can also get the underlying `SearchHit` that Elasticsearch returns
val firstSearchHit = results.searchHits.first()
.apply {
firstSearchHit// this would be useful if we wanted to do some bulk updates
("Hit: $id $seqNo $primaryTerm\n$sourceAsString")
println}
// or we can get both as a `Pair`
val firstHit = results.hits.first()
val (searchHit, deserialized) = firstHit
("Hit: ${searchHit.id}:\n$deserialized") println
Captured Output:
Found 3 hits
Thing(name=The quick brown fox, amount=42)
Thing(name=The quick brown emu, amount=42)
Thing(name=The quick brown gnu, amount=42)
Hit: 1 -2 0
{"name":"The quick brown fox","amount":42}
Hit: 1:
Thing(name=The quick brown fox, amount=42)
We provide several alternative ways to query elasticsearch; including the Kotlin DSL that we used above, raw json in the form of multi line strings, or the Java builders that come with the Java client. For documentation for that see Kotlin Query DSL
Count
We can also query just to get a document count.
("The total number of documents is ${repo.count()}")
println
val text = "quick"
val count = repo.count {
{
configure // instead of "name" you can also use a property reference
= match(Thing::name, text)
query }
}
("We found $count results matching $text") println
Captured Output:
The total number of documents is 100
We found 3 results matching quick
Using multi line strings
Using the Kotlin DSL is nice if you want to programmatically construct your queries in a typesafe way.
However, sometimes you just want to run a query straight from the Kibana Development console in json form. And since Kotlin has multi line strings, doing this is easy.
val results = repo.search {
// we can use Kotlin's string templating
val text = "brown"
("""
source {
"size": 10,
"query": {
"match": {
// did you know ES allows comments in JSON?
// Look we can inject our variable
// but of course beware script injection!
"name": "$text"
}
}
}
""".trimIndent())
}
("Found ${results.totalHits}") println
Captured Output:
Found 3 hits
Multi Search (msearch)
We also have a DSL for msearch
that fires off multiple queries in one go.
val mSearchResults = repo.mSearch {
// a header with a custom searchType
{
header = SearchType.dfs_query_then_fetch
searchType } withQuery {
= matchAll()
query }
// an empty header
{ } withQuery {
header = matchAll()
query }
// adds an empty header
{
withQuery = matchAll()
query }
}
("${mSearchResults.took}ms. and " +
println"returned ${mSearchResults.responses.size} sets of results")
Captured Output:
7ms. and returned 3 sets of results
Scrolling searches
Elasticsearch has a notion of scrolling searches for retrieving large amounts of documents from an index. Normally this works by keeping track of a scroll token and passing that to Elasticsearch to fetch subsequent pages of results. Scrolling is useful if you want to process large amounts of results.
To make scrolling easier and less tedious, the search method on the repository has a simpler solution: simply set scrolling
to true
.
A classic use case for using scrolls is to bulk update your documents. You can do this as follows.
.bulk {
repo// simply set scrolling to true will allow us to scroll over the entire index
// this will scale no matter what the size of your index is. If you use
// scrolling, you can also set the ttl for the scroll (default is 1m)
val results = repo.search(
= true,
scrolling = 10
scrollTtlInMinutes ) {
{
configure // the page size for the scrolling search
// note, resultSize is translated to size. But since size is also
// a function on Map, we work around this.
= 5
resultSize = matchAll()
query }
}
// lets not print lots of results
.hits.take(15).forEach { (hit, thing) ->
results// if you turn off source on your mapping, thing could be null
("${hit.id}: ${thing?.name}")
println}
// after the last page of results, the scroll is cleaned up
}
Captured Output:
1: The quick brown fox
2: The quick brown emu
3: The quick brown gnu
4: Another thing
5: Another thing: 5
6: Another thing: 6
7: Another thing: 7
8: Another thing: 8
9: Another thing: 9
10: Another thing: 10
11: Another thing: 11
12: Another thing: 12
13: Another thing: 13
14: Another thing: 14
15: Another thing: 15