Setting up Full Text Search in PostgreSQL on Ubuntu Server


Using Full Text Search (FTS) you can create a powerful search engine without presenting additional dependencies on extra advanced tools. In this instructional exercise, we'll utilize PostgreSQL to store information containing articles for a speculative news site, at that point figure out how to query the database utilizing FTS and select just the best matches.

1st Step - Creating Example Data

To begin, we'll need a few information to test the full-content search module with, so how about we make some illustration information. On the off chance that you have your own particular table with content values as of now, you can skip to Step 2 and make appropriate substitutions while following along.

in circumstances, the initial step is to connect with the PostgreSQL database from its web server. Since you are associating from a similar host, naturally, you won't have to enter your secret key.

$ sudo -u postgres psql john

This will set up an intuitive PostgreSQL session demonstrating the database name you are working on, which for our situation is john. You should see a john=# database command prompt.


Next, make an illustration table in the database called news. Every section in this table will speak to a news article with a title, some substance, and the writer's name alongside an exceptional identifier.

john=# CREATE TABLE news (


john=#   title TEXT NOT NULL,

john=#  content TEXT NOT NULL,

john=#  author TEXT NOT NULL

john=# );


id is the table's essential list with the unique sort SERIAL, which makes an auto-augment counter for the table. This is a special identifier which consequently goes to the database record. We are going to talk about this index on 3rd step when we flash on performance enhancements.

After that, add some illustration information to the table utilizing the INSERT command. This illustration information in the command underneath speaks to some example news articles.

john=# INSERT INTO news (id, title, content, author name) VALUES

john=#   (1, 'Indian cloud hosting companies', 'Currently there are only few options available for cloud hosting india.', 'bravo'),

john=#   (2, 'Tips and tricks to find best hosting solution', 'Best tips and tricks to find best suitable hosting plans for your website.', Dev),

Since the database has a few information to search for, we can take a stab at keeping in touch with a few questions.


2nd Step - Preparing & Searching Documents


The initial step here is building one report with numerous content sections from the database table. At that point, we can change the subsequent string into a vector of words, which is the thing that we will use in the queries.


johndb=# \x


Expanded display is on.


Initially, we'll have to assemble every one of the columns utilizing the PostgreSQL connect work || and change function to_tsvector().


john=# SELECT title || '. ' || content as document, to_tsvector(title || '. ' || content) as metadata FROM news WHERE id = 1;

You may see that there are less words in the changed form, metadata in the yield above, than in the first report. A portion of the words are unique and each word has a semicolon and a number appended to it. This is on the grounds that the function to_tsvector() standardizes each word to permit to us to discover variation types of a similar word, at that point sorts the outcome one after another in order. The number is the word's situation in the document. There might be extra comma-isolated positions if the standardized word seems more than once.

Presently we can utilize this changed document to exploit the FTS abilities via scanning for the expression "Explorations".

sammy=# SELECT * FROM news WHERE to_tsvector(title || '. ' || content) @@ to_tsquery('Explorations');


The @@ administrator recognizes if the tsvector combines the tsquery or another tsvector. It returns genuine or false, which makes it simple to use as a feature of WHERE criteria.


-[ RECORD 1 ]-----------------------------------------------------
id      | 2
title   | Hitting the shoreline was voted the best piece of life in the locale

content | Investigating tracks and trails was second most mainstream, trailed by going by the shops and after that looking at nearby stops.
author  | Bravo

3rd Step - Enhancing FTS Performance

Generating a report each time we utilize a FTS inquiry can turn into an execution issue when utilizing extensive datasets or little servers. One great answer for this, which we'll execute here, is to create the changed record while embeddings the line and store it alongside the other information. Along these lines, we can simply recover it with a question as opposed to generating it without fail.

To start with, make an additional column called document to the current news table.

john=# ALTER TABLE news ADD "document" tsvector;


We will now need to utilize an alternate query to insert information into the table. Not at all like Step 2, here we will additionally need to set up the changed document and include it into the new report section, this way:

john=# INSERT INTO news (id, title, content, author, document)

john=# VALUES (4, 'Sleep deprivation curing depression', 'Clinicians have long known that there is a strong link between sleep, sunlight and mood.', 'Patel', to_tsvector('Sleep deprivation curing depression' || '. ' || 'Clinicians have long known that there is a strong link between sleep, sunlight and mood.'));


Adding another segment to the current table expects us to include purge esteems for the report section at first. Presently we have to refresh it with the generated esteems.

Utilize the UPDATE command to add the missing information.

john=# UPDATE news SET document = to_tsvector(title || '. ' || content) WHERE document IS NULL;

Adding these rows to the table is a decent execution change, however in vast datasets, we may in any case have issues on the grounds that the database will even now need to check the whole table to discover the lines coordinating the search criteria. A simple answer for this is to utilize files.


At last, indexes enable the database to discover rows quicker via searching utilizing exceptional information structures and algorithms. PostgreSQL has a few kinds of indexes which are suited to specific sorts of questions. The most important ones for this utilization case are GiST files and GIN files. The primary contrast between them is the means by which quick they can recover records from the table. GIN is slower to create while including new information, however speedier to query; GIST creates quicker, yet requires extra information reads.

Since GiST is around 3 times slower to retrieve information than GIN, we will make a GIN record here.

john=# CREATE INDEX idx_fts_search ON news USING gin(document);


Utilizing the indexed document section, our SELECT query has additionally turned into more straightforward.

john=# SELECT title, content FROM news WHERE document @@ to_tsquery('Travel | Cure');


The result will look similar to this:


-[ RECORD 1 ]-----------------------------------------------------
title   | Lack of sleep curing gloom
content | Clinicians have long realized that there is a solid connection between rest, daylight and state of mind.
-[ RECORD 2 ]-----------------------------------------------------
title   | Pacific Northwest fast rail line
content | At present there are just a couple of choices for venturing to every part of the 140 miles amongst Seattle and Vancouver and none of them are perfect.

When you're set, you can leave the database console with \q command.