Searchkick and Elasticsearch

#65 Searchkick and Elasticsearch
1/29/2017

Summary

Add full text searching using Searchkick and Elasticsearch. Here I will show the steps involved in adding this search to an existing application and a sample of autocomplete functionality.
9
rails search performance

Summary

Gemfilegem 'searchkick'
models/movie.rbclass Movie < ApplicationRecord
  searchkick word_start: [:title, :plot]

  def search_data
    {
      title: title,
      year: year,
      plot: plot
    }
  end
    
end

Reindex your model with

Terminal# Terminal

rake searchkick:reindex CLASS=Movie

# irb

Model.reindex
movies_controller.rb  def index
    search = params[:term].present? ? params[:term] : nil
    @movies = if search
      # Movie.where("title LIKE ? OR plot LIKE ?", "%#{search}%", "%#{search}%")
      Movie.search(search, where: { year: { gt: 2000 } })
    else
      Movie.all
    end
  end

  def autocomplete
    render json: Movie.search(params[:query], {
      fields: ["title^5", "plot"],
      match: :word_start,
      limit: 10,
      load: false,
      misspellings: {below: 5}
    }).map(&:title)
  end
movies.js$(document).on('turbolinks:load', function(){
  var movies = new Bloodhound({
    datumTokenizer: Bloodhound.tokenizers.whitespace,
    queryTokenizer: Bloodhound.tokenizers.whitespace,
    remote: {
      url: '/movies/autocomplete?query=%QUERY',
      wildcard: '%QUERY'
    }
  });
  $('#movies_search').typeahead(null, {
    source: movies
  });
})
movies.cssspan.twitter-typeahead .tt-menu,
span.twitter-typeahead .tt-dropdown-menu {
  position: absolute;
  top: 100%;
  left: 0;
  z-index: 1000;
  display: none;
  float: left;
  min-width: 160px;
  padding: 5px 0;
  margin: 2px 0 0;
  list-style: none;
  font-size: 14px;
  text-align: left;
  background-color: #ffffff;
  border: 1px solid #cccccc;
  border: 1px solid rgba(0, 0, 0, 0.15);
  border-radius: 4px;
  -webkit-box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
  box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
  background-clip: padding-box;
}
span.twitter-typeahead .tt-suggestion {
  display: block;
  padding: 3px 20px;
  clear: both;
  font-weight: normal;
  line-height: 1.42857143;
  color: #333333;
  white-space: nowrap;
}
span.twitter-typeahead .tt-suggestion.tt-cursor,
span.twitter-typeahead .tt-suggestion:hover,
span.twitter-typeahead .tt-suggestion:focus {
  color: #ffffff;
  text-decoration: none;
  outline: 0;
  background-color: #337ab7;
}
.input-group.input-group-lg span.twitter-typeahead .form-control {
  height: 46px;
  padding: 10px 16px;
  font-size: 18px;
  line-height: 1.3333333;
  border-radius: 6px;
}
.input-group.input-group-sm span.twitter-typeahead .form-control {
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 3px;
}
span.twitter-typeahead {
  width: 100%;
}
.input-group span.twitter-typeahead {
  display: block !important;
  height: 34px;
}
.input-group span.twitter-typeahead .tt-menu,
.input-group span.twitter-typeahead .tt-dropdown-menu {
  top: 32px !important;
}
.input-group span.twitter-typeahead:not(:first-child):not(:last-child) .form-control {
  border-radius: 0;
}
.input-group span.twitter-typeahead:first-child .form-control {
  border-top-left-radius: 4px;
  border-bottom-left-radius: 4px;
  border-top-right-radius: 0;
  border-bottom-right-radius: 0;
}
.input-group span.twitter-typeahead:last-child .form-control {
  border-top-left-radius: 0;
  border-bottom-left-radius: 0;
  border-top-right-radius: 4px;
  border-bottom-right-radius: 4px;
}
.input-group.input-group-sm span.twitter-typeahead {
  height: 30px;
}
.input-group.input-group-sm span.twitter-typeahead .tt-menu,
.input-group.input-group-sm span.twitter-typeahead .tt-dropdown-menu {
  top: 30px !important;
}
.input-group.input-group-lg span.twitter-typeahead {
  height: 46px;
}
.input-group.input-group-lg span.twitter-typeahead .tt-menu,
.input-group.input-group-lg span.twitter-typeahead .tt-dropdown-menu {
  top: 46px !important;
}


Photo
itscool sid said 3 months ago:

What about on production level ? Elasticsearch consumes memory. How can I manage it on a 2gb aws ubuntu instance?   

635114?v=3&s=64
kobaltz said 3 months ago:

I'd recommend using https://aws.amazon.com/elasticsearch-service if hosting your instances in AWS. Just be mindful and lock the security of the search instance/cluster to the EC2 instance.

Photo
Denny Deng said 3 months ago:

curl http://localhost:9200

I got "unknown regexp options - lcalht", however, elasticsearch.log shows I had started elasticsearch fine.

what did I do wrong?

Thanks in advance

Photo
Denny Deng said 3 months ago:

never mind of my previous comment...forget it. Sorry!

Photo
Denny Deng said 3 months ago:

Hello,

I have cloned your code to my ubuntu 16.04 running Ruby 2.3.3 with rails 5.0.1.

Everything works except autocomplete. I got a 500 Internal Server Error.

The console in Chrome browser showed that it got a stop at line 9537 of Jquery3, It stops at the same line no mater Jquery, Jquery2, or Jquery3 

The line shows the following code as,

xhr.send( options.hasContent && options.data || null );

Any hints for me?

Thanks in advance!



Photo
Denny Deng said 3 months ago:

elasticsearch works in my server. However, I am not sure if I need to configure it for this tutorial.

635114?v=3&s=64
kobaltz said 3 months ago:

Error 500 indicates that something went wrong on the server side and it should have logged something into your application logs. Check there and if you're unable to figure it out, post the relevant section of the log please.

Photo
Denny Deng said 3 months ago:

Sorry, I did not know the format would mess up like this. My email is [email protected] I will send you a email about the log message if you give your email address.

Photo
Denny Deng said 3 months ago:

http://localhost:3000 works

As soon as I typed letter "i", autocomplete started to give me  some error messages.

After "iron" and search, then

Movie.search(search, where: { year: { gt: 2000 } })

is highlighted to indicate where it stopped. What is your Ruby version? I got 2.3.3, perhaps I should try your Ruby version. Thank you for looking into this!

def index
  search = params[:term].present? ? params[:term] : nil
  @movies = if search
    # Movie.where("title LIKE ? OR plot LIKE ?", "%#{search}%", "%#{search}%")
    Movie.search(search, where: { year: { gt: 2000 } })
  else
    Movie.all
  end
end
Searchkick::InvalidQueryError - [400] {"error": {"root_cause": [{"type": "query_shard_exception","reason": "[match] analyzer [searchkick_word_search] not found","index_uuid": "3I4KbiHfSBqIYPWZlzkFKA","index": "movies_development"}], "type": "search_phase_execution_exception","reason": "all shards failed","phase": "query","grouped": true,"failed_shards": [{"shard": 0,"index": "movies_development", "node": "OvOTxtcQQpusc1OQk8tvMQ","reason": {"type": "query_shard_exception","reason": "[match] analyzer [searchkick_word_search] not found","index_uuid": "3I4KbiHfSBqIYPWZlzkFKA","index": "movies_development"}}],"caused_by": {"type": "query_shard_exception","reason": "[match] analyzer [searchkick_word_search] not found","index_uuid": "3I4KbiHfSBqIYPWZlzkFKA","index": "movies_development"}}, "status": 400}: 

635114?v=3&s=64
kobaltz said 3 months ago:

Have you tried reindexing your Movie model?

You can within the console run

# BASH
rails c
# RAILS CONSOLE
Movie.reindex


or straight from Bash

rake searchkick:reindex CLASS=Movie

Photo
Denny Deng said 3 months ago:

I cannot believe I missed reindexing. Thanks a lot! Everything is working now.

635114?v=3&s=64
kobaltz said 3 months ago:

Sweet! Glad that worked!

Photo
Thato Sello said 27 days ago:

This is a great tut and it's much appreciated. 
I would love to know how I could customize the typeahead to include users profile pictures alongside the name to the queried results.

635114?v=3&s=64
kobaltz said 16 days ago:

I think that would be a fun episode to cover and definitely enough content to be its own episode.

Photo
Thato Sello said 10 days ago:

That would be extremely informative and thank you for the response kobaltz. 

Login to Comment