First experience using Redis

Surely, I have heard about Redis before but I didn't use it in any project. So when I thought of using it I kept on making a typo. I was writing Reddis instead of Redis. Yes, I realized that I was using an extra d in the word. But now I can write the word perfectly, let me share with you my first time using this database.

What is Redis?

Yes, we need to start with this question to get a deep understanding of the things we will be discussing further on. Basically, Redis is an open-source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. This definition is according to the official website itself.

I haven't used their all services but if I have to answer this question from my own experience then I will probably say Redis is a database that facilitates us to store the data in memory in JSON format and also enables us to intelligently search for the data stored inside of the database. I mean their search feature is so cool that it even returns us the result that matches the synonyms of our query, of course only if we ask for it.

Why is it popular?

From the StackOverflow survey of the different years, I have prepared the following data where we can clearly see the year-on-year growth of Redis amongst the professional developers.

Redis favourite.png

We can clearly see that the use of Redis by professional developers in the year

As other popular databases like MySQL, PostgreSQL, and MongoDB have insane use amongst professional developers. The Redis is also catching up with them slowly. But one of the reasons why Redis is insanely popular is because it tops the list of most loved databases. Yes, I am speaking here on the basis of the same survey. Let me plot out the data here:

I think I also love Redis. But I am not one of those developers that took part in this survey.

Why I chose Redis

Of course, I wouldn't have chosen Redis in the middle of nowhere. I was already pretty comfortable with MongoDB so there is a certain condition that triggered me to go with this database. And that condition seems to be a recent hackathon conducted by Redis and Dev.to on August.

I have talked about the hackathon so now I have to share my submission with you, right? And my submission is a search engine. Yes, I will definitely write an article on how I made a search engine and what things to be taken into consideration while making your search engine.

How I used Redis

I used Redis just like any other database. Yes, I used it to literally save the data. The in-memory database provided by Redis is so powerful and fast enough to power my search engine that I kind of fell in love with it. I scraped websites using a bot and saved the main contents of the website inside the Redis database. Then whenever the user was searching for the query, the query was sent to the backend from where the results related to the query were fetched from the database using Redis Search.

I don't want to get too techy here and also I don't want to throw lots of technical jargon but we can take a look at the database schema we created, okay?

How to CRUD

I have used Redis with Nodejs and there is a redis-om package for nodejs that helps users to use the Redis database either on the local machine or the database on the Redis cloud itself.

Connecting with database

import { Client } from "redis-om";

const client = new Client();

const connectDB = async () => {
  if (!client.isOpen()) {
    try {
      await client.open(process.env.REDIS_WEBSITE_URL);
      console.log("Connected to redis !");
    } catch (error) {
      console.log("failed connecting redis", error);
    }
  }
};

Here I imported the Client from redis-om which helps us to connect our server with the database. Since I have my Redis database hosted on the Redis cloud, we get a public endpoint that we can use to create a connection URL and I have saved that URL inside of the .env file. Of course, there is a security concern, ladies and gentlemen. After which I created a function that checks if we are already connected with the database or not. If we are not connected then it connects us with the database.

Yes, connecting with the database was that simple. Now let's create a schema. Why do we need a schema? We need schema because both the database and the developers need to know what are the properties of which type they are dealing with. On top of it, Redis have different methods for different databases.

Create a schema

We need to have an entity before creating the schema so we will be importing both Entity and Schema from redis-om. Then we will create a class for our entity, in this case, I am storing the data of a website so I named the entity as a website. And the website extends the Entity or in other words website inherits the Entity class.

Then I created and exported the Schema of the website. Inside of the schema, I have defined different fields with their data type.

import { Entity, Schema } from "redis-om";

class Website extends Entity {}

export const websiteSchema = new Schema(
  Website,
  {
    url: {
      type: "string",
    },
    title: {
      type: "text",
      textSearch: true,
    },
    description: {
      type: "text",
      textSearch: true,
    },
    firstFewWords: {
      type: "string",
    },
    loadTime: {
      type: "number",
    },
    lastUpdated: {
      type: "date",
    },
    backLinks: {
      type: "number",
    },
    backLinkKeywords: {
      type: "string[]",
    },
    urlKeywords: {
      type: "string[]",
    },
    mainKeywords: {
      type: "string[]",
      textSearch: true,
    },
    headings: {
      type: "string[]",
      textSearch: true,
    },
    favicon: {
      type: "string",
    },
    ogImage: {
      type: "string",
    },
  },
  {
    dataStructure: "JSON",
  }
);

I have used fields with different data types here such as a string, number, text, and string array. If you want to get more of the idea of creating a schema, you can find official documentation. Just in case, if you are wondering about what dataStructure: "JSON" means in the above code then it is there because we are using a RedisJSON database. And the textSearch: true property inside of the field will help us during the search of the result inside the Redis database.

Let's save something

There is not much of a complex thing when it comes to saving the data. Here, I have created a function that connects with the database by calling the function that we created before. Then it creates the repository out of the websiteSchema we created above. After that a new entity is created and saved with the data we got from our parameter.

export const saveWebsiteData = async (website) => {
  await connectDB();
  const repository = client.fetchRepository(websiteSchema);  
  const site = repository.createEntity(website);
  const id = await repository.save(site);

  return id;
};

Yes, it was this simple to save the data. Other operations are also simple to perform. I think it is more of the personal experience I had with Redis than the tutorial on how to do things. Again, you can look at the official documentation on how to perform the CRUD.

Resources

I want to share some resources that I found quite helpful in my journey of learning and implementing Redis. One of the beginner-friendly tutorials is from WebDevSimplified where he goes over the basics of Redis. And then I found a video from Firebase where he actually shows to create API with Redis using Next js and he also covers how to use Redis Search. It is a short tutorial but has a lot of things to take away. Here is the video

https://www.youtube.com/watch?v=DOIWQddRD5M

I will leave some resources here:

Problem

How can I end some conversation without discussing the problems? Of course, I faced too many problems. I have overcome some problems but I am not able to find solutions for some problems to this date. Please share with me the answer to the problems that I am going to lay out here.

While saving the data, redundant data was being saved. Two entities having the same field should not exist. Yes, I wanted a unique field of the entity except the entityId itself. But I wasn't able to find a solution where I could have entities with unique field values. If I have to provide a reference then while creating a model using mongoose, I can set a unique property.
While I was trying to update an entity using other fields, I wasn't able to update the entity. I searched across the internet but I found only the solutions where entityId is required to update that entity. So, to update the entity with a certain field value, I had to use Redis Search first to get all of the entities with matching field value then I would take the entityId from there and use that to update the entity.

Alright, I was able to remember only these two problems. I would be so happy if anyone could share the solution to the above problems.

Summary

To summarize my experience, I had to get started with Redis in order to build a project for a hackathon. I was able to find some really good tutorials on YouTube explaining some of the things that I wanted and also some of the things that I truly didn't want. Also, I had a bit more problems searching for the solution to the Redis-related bug and errors. I wasn't able to get the right result to solve the problem. The official documentation surely helped me out but it was either too short or the database itself has limited methods.

Anyway, the redis insight helped me to see the data in a structured format in a GUI. I fell in love with the Redis search for sure but I think they should work more on making CRUD operations easier to deal with.

If you have anything to share or have the answer to my question then you can comment below or ping me up on Twitter.