Searchkick and Elasticsearch

#65 Searchkick and Elasticsearch
1/29/2017

Summary

Add full text searching using Searchkick and Elasticsearch. Here I will show the steps involved in adding this search to an existing application and a sample of autocomplete functionality.
7
rails search performance

Summary

Gemfilegem 'searchkick'
models/movie.rbclass Movie < ApplicationRecord
  searchkick word_start: [:title, :plot]

  def search_data
    {
      title: title,
      year: year,
      plot: plot
    }
  end
    
end

Reindex your model with

Terminal# Terminal

rake searchkick:reindex CLASS=Movie

# irb

Model.reindex
movies_controller.rb  def index
    search = params[:term].present? ? params[:term] : nil
    @movies = if search
      # Movie.where("title LIKE ? OR plot LIKE ?", "%#{search}%", "%#{search}%")
      Movie.search(search, where: { year: { gt: 2000 } })
    else
      Movie.all
    end
  end

  def autocomplete
    render json: Movie.search(params[:query], {
      fields: ["title^5", "plot"],
      match: :word_start,
      limit: 10,
      load: false,
      misspellings: {below: 5}
    }).map(&:title)
  end
movies.js$(document).on('turbolinks:load', function(){
  var movies = new Bloodhound({
    datumTokenizer: Bloodhound.tokenizers.whitespace,
    queryTokenizer: Bloodhound.tokenizers.whitespace,
    remote: {
      url: '/movies/autocomplete?query=%QUERY',
      wildcard: '%QUERY'
    }
  });
  $('#movies_search').typeahead(null, {
    source: movies
  });
})
movies.cssspan.twitter-typeahead .tt-menu,
span.twitter-typeahead .tt-dropdown-menu {
  position: absolute;
  top: 100%;
  left: 0;
  z-index: 1000;
  display: none;
  float: left;
  min-width: 160px;
  padding: 5px 0;
  margin: 2px 0 0;
  list-style: none;
  font-size: 14px;
  text-align: left;
  background-color: #ffffff;
  border: 1px solid #cccccc;
  border: 1px solid rgba(0, 0, 0, 0.15);
  border-radius: 4px;
  -webkit-box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
  box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
  background-clip: padding-box;
}
span.twitter-typeahead .tt-suggestion {
  display: block;
  padding: 3px 20px;
  clear: both;
  font-weight: normal;
  line-height: 1.42857143;
  color: #333333;
  white-space: nowrap;
}
span.twitter-typeahead .tt-suggestion.tt-cursor,
span.twitter-typeahead .tt-suggestion:hover,
span.twitter-typeahead .tt-suggestion:focus {
  color: #ffffff;
  text-decoration: none;
  outline: 0;
  background-color: #337ab7;
}
.input-group.input-group-lg span.twitter-typeahead .form-control {
  height: 46px;
  padding: 10px 16px;
  font-size: 18px;
  line-height: 1.3333333;
  border-radius: 6px;
}
.input-group.input-group-sm span.twitter-typeahead .form-control {
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 3px;
}
span.twitter-typeahead {
  width: 100%;
}
.input-group span.twitter-typeahead {
  display: block !important;
  height: 34px;
}
.input-group span.twitter-typeahead .tt-menu,
.input-group span.twitter-typeahead .tt-dropdown-menu {
  top: 32px !important;
}
.input-group span.twitter-typeahead:not(:first-child):not(:last-child) .form-control {
  border-radius: 0;
}
.input-group span.twitter-typeahead:first-child .form-control {
  border-top-left-radius: 4px;
  border-bottom-left-radius: 4px;
  border-top-right-radius: 0;
  border-bottom-right-radius: 0;
}
.input-group span.twitter-typeahead:last-child .form-control {
  border-top-left-radius: 0;
  border-bottom-left-radius: 0;
  border-top-right-radius: 4px;
  border-bottom-right-radius: 4px;
}
.input-group.input-group-sm span.twitter-typeahead {
  height: 30px;
}
.input-group.input-group-sm span.twitter-typeahead .tt-menu,
.input-group.input-group-sm span.twitter-typeahead .tt-dropdown-menu {
  top: 30px !important;
}
.input-group.input-group-lg span.twitter-typeahead {
  height: 46px;
}
.input-group.input-group-lg span.twitter-typeahead .tt-menu,
.input-group.input-group-lg span.twitter-typeahead .tt-dropdown-menu {
  top: 46px !important;
}


Photo
itscool sid said about 1 month ago:

What about on production level ? Elasticsearch consumes memory. How can I manage it on a 2gb aws ubuntu instance?   

635114?v=3&s=64
kobaltz said about 1 month ago:

I'd recommend using https://aws.amazon.com/elasticsearch-service if hosting your instances in AWS. Just be mindful and lock the security of the search instance/cluster to the EC2 instance.

Photo
Denny Deng said 19 days ago:

curl http://localhost:9200

I got "unknown regexp options - lcalht", however, elasticsearch.log shows I had started elasticsearch fine.

what did I do wrong?

Thanks in advance

Photo
Denny Deng said 19 days ago:

never mind of my previous comment...forget it. Sorry!

Photo
Denny Deng said 19 days ago:

Hello,

I have cloned your code to my ubuntu 16.04 running Ruby 2.3.3 with rails 5.0.1.

Everything works except autocomplete. I got a 500 Internal Server Error.

The console in Chrome browser showed that it got a stop at line 9537 of Jquery3, It stops at the same line no mater Jquery, Jquery2, or Jquery3 

The line shows the following code as,

xhr.send( options.hasContent && options.data || null );

Any hints for me?

Thanks in advance!



Photo
Denny Deng said 19 days ago:

elasticsearch works in my server. However, I am not sure if I need to configure it for this tutorial.

635114?v=3&s=64
kobaltz said 19 days ago:

Error 500 indicates that something went wrong on the server side and it should have logged something into your application logs. Check there and if you're unable to figure it out, post the relevant section of the log please.

Photo
Denny Deng said 18 days ago:

Sorry, I did not know the format would mess up like this. My email is [email protected] I will send you a email about the log message if you give your email address.

Photo
Denny Deng said 18 days ago:

http://localhost:3000 works

As soon as I typed letter "i", autocomplete started to give me  some error messages.

After "iron" and search, then

Movie.search(search, where: { year: { gt: 2000 } })

is highlighted to indicate where it stopped. What is your Ruby version? I got 2.3.3, perhaps I should try your Ruby version. Thank you for looking into this!

def index
  search = params[:term].present? ? params[:term] : nil
  @movies = if search
    # Movie.where("title LIKE ? OR plot LIKE ?", "%#{search}%", "%#{search}%")
    Movie.search(search, where: { year: { gt: 2000 } })
  else
    Movie.all
  end
end
Searchkick::InvalidQueryError - [400] {"error": {"root_cause": [{"type": "query_shard_exception","reason": "[match] analyzer [searchkick_word_search] not found","index_uuid": "3I4KbiHfSBqIYPWZlzkFKA","index": "movies_development"}], "type": "search_phase_execution_exception","reason": "all shards failed","phase": "query","grouped": true,"failed_shards": [{"shard": 0,"index": "movies_development", "node": "OvOTxtcQQpusc1OQk8tvMQ","reason": {"type": "query_shard_exception","reason": "[match] analyzer [searchkick_word_search] not found","index_uuid": "3I4KbiHfSBqIYPWZlzkFKA","index": "movies_development"}}],"caused_by": {"type": "query_shard_exception","reason": "[match] analyzer [searchkick_word_search] not found","index_uuid": "3I4KbiHfSBqIYPWZlzkFKA","index": "movies_development"}}, "status": 400}: 

635114?v=3&s=64
kobaltz said 18 days ago:

Have you tried reindexing your Movie model?

You can within the console run

# BASH
rails c
# RAILS CONSOLE
Movie.reindex


or straight from Bash

rake searchkick:reindex CLASS=Movie

Photo
Denny Deng said 18 days ago:

I cannot believe I missed reindexing. Thanks a lot! Everything is working now.

635114?v=3&s=64
kobaltz said 18 days ago:

Sweet! Glad that worked!

Login to Comment