Arsalan Shahid

Implement Full-text Search in Django using Elasticsearch – StackDuplica Part 3

My dear reader, how are you? السلام عليكم

“Find joy in the journey.” Thomas S. Monson

This is the third part of StackDuplica web-Application tutorial series. We will build up a full-text search-engine for questions using elasticsearch.

Few useful links to practically follow the project:

  1. StackDuplica GitHub repository — DirectMe
  2. All other tutorials on StackDuplica — DirectMe
  3. Set your GitHub repo to StackDuplica Part 2 using the following command:
git fetch origin 9b4f6b8a3330399f47aeeb8b86b76202aa94a68a

Start by Setting-up Elasticsearch

Elasticsearch is a search engine based on the Lucene library and provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. It is developed in Java and is open-source under various open-source licenses (mostly the Apache License).

(stackenv)StackDuplica$ sudo apt-get update
(stackenv)StackDuplica$ sudo apt install curl 

# Obtain Elasticsearch image and run service using docker as shown below 

(stackenv)StackDuplica$ sudo docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node"

# Create an index for qanda application 

(stackenv)StackDuplica$ curl -XPUT "localhost:9200/qanda?pretty"

Let us now create an elasticsearch service code that will load the list of questions in the database as shown below

# Create qanda/service/ and add the ollowing program into it

from elasticsearch import Elasticsearch
from elasticsearch.helpers import streaming_bulk

from django.conf import settings

import logging

ALREADY_EXISTS_EXCEPTION = 'resource_already_exists_exception'

FAILED_TO_LOAD_ERROR = 'Failed to load {}: {!r}'

ISO_DATE_TIME_FORMAT = '%Y-%m-%dT%H:%M:%S.%fZ'

logger = logging.getLogger(__name__)

def get_client():
    return Elasticsearch(hosts=[
        {'host': settings.ES_HOST, 'port': settings.ES_PORT,}

def bulk_load(questions):
    all_ok = True
    es_questions = (q.as_elasticsearch_dict() for q in questions)
    for ok, result in streaming_bulk(
        if not ok:
            all_ok = False
            action, result = result.popitem()
            logger.error(FAILED_TO_LOAD_ERROR.format(result['_id'], result))
    return all_ok

def search_for_questions(query):
    client = get_client()
    result =, body={
      'query': {
          'match': {
              'text': query,
    return (h['_source'] for h in result['hits']['hits'])

def upsert(question_model):
    client = get_client()
    question_dict = question_model.as_elasticsearch_dict()
    doc_type = question_dict['_type']
    del question_dict['_id']
    del question_dict['_type']
    response = client.update(
            'doc': question_dict,
            'doc_as_upsert': True,
    return response

Update Question Model

Once the elastic service code is in place, we will update our questions model as shown below

# Open qanda/ and add the following function in Question model

def as_elasticsearch_dict(self):
        return {
            '_type': 'doc',
            'text': '{}\n{}'.format(self.title, self.question),
            'question_body': self.question,
            'title': self.title,
            'created': self.created,

Install elesticsearch using pip3 and create command to load questions in elasticsearch

Let us not install elasticsearch using pip3 for our Django project. Add elasticsearch==6.00 in and using the following command to install it

(stackenv)StackDuplica$ pip3 install -r

Create a command to load the questions Django models to elasticsearch as shown below

# create qanda/management/commands/ and add the following program into it.

from import BaseCommand

from qanda.service import elasticsearch
from qanda.models import Question

class Command(BaseCommand):
    help = 'Load all questions into Elasticsearch'

    def handle(self, *args, **options):
        queryset = Question.objects.all()
        all_loaded = elasticsearch.bulk_load(queryset)
        if all_loaded:
                'Successfully loaded all questions into Elasticsearch.'))
      'Some questions not loaded '
                                   'successfully. See logged errors')

Load the questions into the elastic search database

(stackenv)StackDuplica$ python3 load_questions_into_elasticsearch

create a search view

# open qanda/ and add the following 

from django.views.generic import TemplateView

class SearchView(TemplateView):
    template_name = 'qanda/search.html'

    def get_context_data(self, **kwargs):
        query = self.request.GET.get('q', None)
        ctx = super().get_context_data(query=query, **kwargs)
        if query:
            results = search_for_questions(query)
            ctx['hits'] = results
        return ctx

Add a route for the search view

# Open qanda/ and add the following

urlpatterns = [
path('q/search', views.SearchView.as_view(), name='question_search'),

Create a template for search view

# create qanda/tempates/qanda/search.html and add the following program 

{% extends "base.html" %}
{% load markdownify %}

{% block body %}
  <h2 >Search</h2 >
  <form method="get" class="form-inline" >
    <input class="form-control mr-2"
           name="q" value="{{ query }}" >
    <button type="submit" class="btn btn-primary" >Search</button >
  </form >
  {% if query %}
    <h3>Results from search query '{{ query }}'</h3 >
    <ul class="list-unstyled search-results" >
      {% for hit in hits %}
        <li >
          <a href="{% url "qanda:question_detail" %}" >
            {{ hit.title }}
          </a >
          <div >
            {{ hit.question_body|markdownify|truncatewords_html:20 }}
          </div >
        </li >
      {% empty %}
        <li >No results.</li >
      {% endfor %}
    </ul >
  {% endif %}
{% endblock %}

# open StackApp/templates/base.html and update the base template as shown below 

{% load static %}
<!DOCTYPE html>
<html lang="en" >
<head >
  <meta charset="UTF-8" >
  <title >{% block title %}StackDuplica{% endblock %}</title >
      crossorigin="anonymous" >
      crossorigin="anonymous" >
  <link rel="stylesheet" href="{% static "base.css" %}" >
</head >
<body >
<nav class="navbar navbar-expand-lg  bg-light" >
  <div class="container" >
    <a class="navbar-brand" href="/" >StackDuplica</a >
    <ul class="navbar-nav" >
	     <li class="nav-item" >
        <a class="nav-link" href="{% url "qanda:ask" %}" >Ask</a >
      </li >
      <li class="nav-item" >
            href="{% url "qanda:index" %}" >
          Today's Questions
        </a >
      </li >

      <li class="nav-item" >
        <form class="form-inline"
              action="{% url "qanda:question_search" %}"
          <input class="form-control mr-sm-2" type="search"
          <button class="btn btn-outline-primary my-2 my-sm-0" type="submit" >
          </button >
        </form >
      </li > 
    </ul >
  </div >
</nav >
<div class="container" >
  {% block body %}{% endblock %}
</div >
</body >
</html >

If you run the development server at this point in time you should be able to see the added functionalities at

I hope you find this tutorial useful. If you find any errors or feel any need for improvement, let me know in your comments below.

Signing off for today. Stay tuned and I will see you next week! Happy learning.

Exit mobile version