#65 Searchkick and Elasticsearch
1-29-2017

Summary

Add full text searching using Searchkick and Elasticsearch. Here I will show the steps involved in adding this search to an existing application and a sample of autocomplete functionality.
15
rails search performance 7:32

Summary

Gemfilegem 'searchkick'
models/movie.rbclass Movie < ApplicationRecord
  searchkick word_start: [:title, :plot]

  def search_data
    {
      title: title,
      year: year,
      plot: plot
    }
  end
    
end

Reindex your model with

Terminal# Terminal

rake searchkick:reindex CLASS=Movie

# irb

Model.reindex
movies_controller.rb  def index
    search = params[:term].present? ? params[:term] : nil
    @movies = if search
      # Movie.where("title LIKE ? OR plot LIKE ?", "%#{search}%", "%#{search}%")
      Movie.search(search, where: { year: { gt: 2000 } })
    else
      Movie.all
    end
  end

  def autocomplete
    render json: Movie.search(params[:query], {
      fields: ["title^5", "plot"],
      match: :word_start,
      limit: 10,
      load: false,
      misspellings: {below: 5}
    }).map(&:title)
  end
movies.js$(document).on('turbolinks:load', function(){
  var movies = new Bloodhound({
    datumTokenizer: Bloodhound.tokenizers.whitespace,
    queryTokenizer: Bloodhound.tokenizers.whitespace,
    remote: {
      url: '/movies/autocomplete?query=%QUERY',
      wildcard: '%QUERY'
    }
  });
  $('#movies_search').typeahead(null, {
    source: movies
  });
})
movies.cssspan.twitter-typeahead .tt-menu,
span.twitter-typeahead .tt-dropdown-menu {
  position: absolute;
  top: 100%;
  left: 0;
  z-index: 1000;
  display: none;
  float: left;
  min-width: 160px;
  padding: 5px 0;
  margin: 2px 0 0;
  list-style: none;
  font-size: 14px;
  text-align: left;
  background-color: #ffffff;
  border: 1px solid #cccccc;
  border: 1px solid rgba(0, 0, 0, 0.15);
  border-radius: 4px;
  -webkit-box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
  box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
  background-clip: padding-box;
}
span.twitter-typeahead .tt-suggestion {
  display: block;
  padding: 3px 20px;
  clear: both;
  font-weight: normal;
  line-height: 1.42857143;
  color: #333333;
  white-space: nowrap;
}
span.twitter-typeahead .tt-suggestion.tt-cursor,
span.twitter-typeahead .tt-suggestion:hover,
span.twitter-typeahead .tt-suggestion:focus {
  color: #ffffff;
  text-decoration: none;
  outline: 0;
  background-color: #337ab7;
}
.input-group.input-group-lg span.twitter-typeahead .form-control {
  height: 46px;
  padding: 10px 16px;
  font-size: 18px;
  line-height: 1.3333333;
  border-radius: 6px;
}
.input-group.input-group-sm span.twitter-typeahead .form-control {
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 3px;
}
span.twitter-typeahead {
  width: 100%;
}
.input-group span.twitter-typeahead {
  display: block !important;
  height: 34px;
}
.input-group span.twitter-typeahead .tt-menu,
.input-group span.twitter-typeahead .tt-dropdown-menu {
  top: 32px !important;
}
.input-group span.twitter-typeahead:not(:first-child):not(:last-child) .form-control {
  border-radius: 0;
}
.input-group span.twitter-typeahead:first-child .form-control {
  border-top-left-radius: 4px;
  border-bottom-left-radius: 4px;
  border-top-right-radius: 0;
  border-bottom-right-radius: 0;
}
.input-group span.twitter-typeahead:last-child .form-control {
  border-top-left-radius: 0;
  border-bottom-left-radius: 0;
  border-top-right-radius: 4px;
  border-bottom-right-radius: 4px;
}
.input-group.input-group-sm span.twitter-typeahead {
  height: 30px;
}
.input-group.input-group-sm span.twitter-typeahead .tt-menu,
.input-group.input-group-sm span.twitter-typeahead .tt-dropdown-menu {
  top: 30px !important;
}
.input-group.input-group-lg span.twitter-typeahead {
  height: 46px;
}
.input-group.input-group-lg span.twitter-typeahead .tt-menu,
.input-group.input-group-lg span.twitter-typeahead .tt-dropdown-menu {
  top: 46px !important;
}


itscool sid said over 1 year ago:

What about on production level ? Elasticsearch consumes memory. How can I manage it on a 2gb aws ubuntu instance?   

kobaltz PRO said over 1 year ago:

I'd recommend using https://aws.amazon.com/elasticsearch-service if hosting your instances in AWS. Just be mindful and lock the security of the search instance/cluster to the EC2 instance.

Denny Deng said over 1 year ago:

curl http://localhost:9200

I got "unknown regexp options - lcalht", however, elasticsearch.log shows I had started elasticsearch fine.

what did I do wrong?

Thanks in advance

Denny Deng said over 1 year ago:

never mind of my previous comment...forget it. Sorry!

Denny Deng said over 1 year ago:

Hello,

I have cloned your code to my ubuntu 16.04 running Ruby 2.3.3 with rails 5.0.1.

Everything works except autocomplete. I got a 500 Internal Server Error.

The console in Chrome browser showed that it got a stop at line 9537 of Jquery3, It stops at the same line no mater Jquery, Jquery2, or Jquery3 

The line shows the following code as,

xhr.send( options.hasContent && options.data || null );

Any hints for me?

Thanks in advance!



Denny Deng said over 1 year ago:

elasticsearch works in my server. However, I am not sure if I need to configure it for this tutorial.

kobaltz PRO said over 1 year ago:

Error 500 indicates that something went wrong on the server side and it should have logged something into your application logs. Check there and if you're unable to figure it out, post the relevant section of the log please.

Denny Deng said over 1 year ago:

Sorry, I did not know the format would mess up like this. My email is [email protected] I will send you a email about the log message if you give your email address.

Denny Deng said over 1 year ago:

http://localhost:3000 works

As soon as I typed letter "i", autocomplete started to give me  some error messages.

After "iron" and search, then

Movie.search(search, where: { year: { gt: 2000 } })

is highlighted to indicate where it stopped. What is your Ruby version? I got 2.3.3, perhaps I should try your Ruby version. Thank you for looking into this!

def index
  search = params[:term].present? ? params[:term] : nil
  @movies = if search
    # Movie.where("title LIKE ? OR plot LIKE ?", "%#{search}%", "%#{search}%")
    Movie.search(search, where: { year: { gt: 2000 } })
  else
    Movie.all
  end
end
Searchkick::InvalidQueryError - [400] {"error": {"root_cause": [{"type": "query_shard_exception","reason": "[match] analyzer [searchkick_word_search] not found","index_uuid": "3I4KbiHfSBqIYPWZlzkFKA","index": "movies_development"}], "type": "search_phase_execution_exception","reason": "all shards failed","phase": "query","grouped": true,"failed_shards": [{"shard": 0,"index": "movies_development", "node": "OvOTxtcQQpusc1OQk8tvMQ","reason": {"type": "query_shard_exception","reason": "[match] analyzer [searchkick_word_search] not found","index_uuid": "3I4KbiHfSBqIYPWZlzkFKA","index": "movies_development"}}],"caused_by": {"type": "query_shard_exception","reason": "[match] analyzer [searchkick_word_search] not found","index_uuid": "3I4KbiHfSBqIYPWZlzkFKA","index": "movies_development"}}, "status": 400}: 

kobaltz PRO said over 1 year ago:

Have you tried reindexing your Movie model?

You can within the console run

# BASH
rails c
# RAILS CONSOLE
Movie.reindex


or straight from Bash

rake searchkick:reindex CLASS=Movie

Denny Deng said over 1 year ago:

I cannot believe I missed reindexing. Thanks a lot! Everything is working now.

kobaltz PRO said over 1 year ago:

Sweet! Glad that worked!

Thato Sello said about 1 year ago:

This is a great tut and it's much appreciated. 
I would love to know how I could customize the typeahead to include users profile pictures alongside the name to the queried results.

kobaltz PRO said about 1 year ago:

I think that would be a fun episode to cover and definitely enough content to be its own episode.

Thato Sello said about 1 year ago:

That would be extremely informative and thank you for the response kobaltz. 

lupeng liu said 12 months ago:

hi, the video is very excellent. But I am a new guy to ruby on rails. I clone the project from github to local. I run `rake db:create` `rake db:migrate` , it seems work well.

but when i run `rake db:seed`, errors comes.  please see the log below, can you help me to solve this problem?


lupeng liu said 12 months ago:

And, when I visit localhost:3000/movies/new,  errors come out.  please see below, can you give some suggestion?


kobaltz PRO said 12 months ago:

The RestClient appears to be returning an Unauthorized response and your seeds file isn’t taking that into consideration. Hope that helps.

Login to Comment