Are you still checking the price of your favorite MacBook or the stock price of Tesla? Have you ever imagined what if anybody takes that pain from you and shoots an email to you when the price is within your budget? Well, just leave that boring process up to me and you just check your email. I have accomplished the thing, introducing https://lootle.live.
Project introduction
Lootle -> (loot-le) is a website and MIT license open source project where you can keep track of the price of your MacBook or of your favorite Raspberry P or the stock price of Microsoft. You just enter the URL of the product/stock then the website keeps the history of the price fluctuation, presents it to you, and even notifies you when the price drops down that you specified.
Note: Some of the most popular sites are supported till now and support for more websites is coming soon.
Inspiration, idea, and problem 🤷♀️
Redundancy is a crime and the person who writes the same code is a criminal. Then why we should repeatedly check the price of something that we want? This isn't something awesome, right? This was the problem that I thought I can solve. And just recently, DevEd published a video showing the use of cheerio. Something happened inside of my mind at that point and after some thought and research on that, here I am with the full-fledged product.
Demo of the website
Homepage
The first page you see when you land on lootle is this page where you are presented with some text, a few buttons, and an awesome video that introduces the website in itself. You can sign in with google for the app in two ways, you can use whatever you like, then there is a button that contains the link of the GitHub repository. And you find a features section and a little footer when social media handles and attributions are provided.
Dashboard app
Here you find two options, not too much but enough to do our stuff. The first one is to create a new tracker and the second one is to view the history of price fluctuation. You find nothing much more complicated and fancier here, just some buttons and text boxes are here to let you use the service.
Other pages
You get a very simple 404 page on the website.
How did I build it
I coded the frontend part in react, created an express server and deployed a bot/scraper on linode.
Tech stack 👩💻
- Reactjs -> For frontend
- MongoDB Atlas -> For mongoDB database
- Express.js -> For API
- Linode -> Deployment
Process 🤔
No, it's not a whole tutorial here on how to make the whole website. Better you can think of it as a general idea and steps. But if like to build one, you can find it here.
1. 🏬 Frontend:
I am quite familier with react so it wasn't that hard to start. I started with create-react-app and mui. And I have written some custom CSS also. I have learned to define routes in new way and manage different states easily.
2. 📝 REST API:
As I was creating frontend, I had to somehow communicate with the backend. So, I created an express server to listen on port 3001 and when I hit different endpoints, I was able to perform different tasks. Some of those actions are:
- Get all trackers
- Get all supported sites
- Get trackers of a specific user
- Update tracker
- Delete tracker and a lot more
3. 🛢️Setting up database
I used mongoDB, if I have to be more specific, I used mongoDB atlas to store the information gathered by the tracker. I made different models and only allowed the server to communicate.
4. 🌐PWA
Lootle is a Progressive Web App, meaning that it can work offline and can be installed, and you can access it by clicking on the icon. I had to make it😝.
5. 🤖Making a scraper
I used cheerio.js and axios to extract data from the website and in every ten minutes. The bot checks the price of the item and shoots an email if the current price of the product is less than the specified price of the user. Here is a scrapper.
Deployment
I am very new to Linode and introduced to their services for the first time. So, it took me quite some research and time to deploy the app. I hosted my MERN app as well as the bot on Linode using an Ubuntu server.
Challenges I faced 💪
- Of course, it was my first time with Linode but it was worth learning
- Work on the terminal and use linux commands
- Get the SSH certificate
- Sending the emails to the user
- And I was in the situation feeling I don't know anything, every day
Functions ⚒
Oh, we haven't discussed much on the functions of the website 😅. It checks the price of the product specified. Not only it checks the price, it extracts the name and information of the product from the website. It gathers information on a regular interval of ten minutes and stores inside of the database. If the current price is found to be less than the specified price by the user then, the user will be updated with an email.
And the user can also check the history of the price fluctuation through the nice little dashboard.
💡 What did I learn?
I have made a few React projects in the past but I haven't made any project this. I had very bad experience with proxies this time but it was all worth it, I learned to use network tab. I learned to use linux a bit more and most importantly I learned how important skill learning linux is. There were a lot of googling, frustrating moments, and bugs without errors. But I surface summarize my outcomes here as follows:
- Using Linode: From complete beginner to being able to use its some of the amazing features.
- Scraping Being able to scrape the data from different sites and format that data with regular expressions.
Important links 🔗
- Github repo of MERN app
- GitHub repo of the scrapper
- Website
- Dashboard
Currently supported sites:
- Flipkart.com for available products only
- Amazon.com for available products only
- cnbc.com for stock price
Summary
Overall the hackathon was a great exprience for learning. I got to learn about linux, network tab in developer console, learned some better way to google. And most important of all, I developed a new skill of being patient when my code doesn't work and I don't get any solutions at all. Taking help from friends on discord and shaping the app, the journey was awesome. If you haven't tried yet, try it now, it is mobile responsive.
Connect with me on Twitter @aashishpanthi11.
