Skip to content
Episteme

Adding Posts to Database

After making sure the posts which were scraped aren’t already in the database I have to save said posts in the database. There’s essentially two things that I have to save to the database:

  1. Our analyzed theses (Points & Criticisms the AI extracted from the posts)
  2. The post info which will be linked to the analyzed data so that the app can later show the source of each point/criticism (URL of the post, title, author, etc.)

The order doesn’t really matter too much although I decided to essentially first save the posts to the DB.

However remember this is the first time I’m saving anything proper to the database from the script which means I’ll first have to do some setup work. At the start of the flow (after receiving the task to do an analysis on a given ticker) the first thing I’ll do is check if that ticker is already in the database. If it isn’t I’ll first have to create a new Ticker entry in the DB.

The script will create a new ticker, setting the symbol to the ticker and retrieving the name of the stock by looking the ticker up in the stocks database I implemented earlier to suggest stocks to the user.

After scraping the posts the app will go through each post’s info and save it to the database, linking it to the ticker. The program flow will keep track of all posts primary id’s after committing them to the database. This way it can then again pull the new posts info from the database and pass said info to the analysis module in the next step.