Project 10: Build a Chatbot with LangChain and Chroma to chat with your own documents

Project 9: Generative QA with Retrieval-Augmented Generation (RAG) and TruEra Evaluation

Project 8: Neural Machine Translation

Project 7: Extractive Question Answering

Project 6: Natural Language Inference & Explainable AI

Project 5: Anomaly Detection

Project 4: Image Classification & Explainable AI

Project 3: Time Series

Project 2: Logistic Regression

Project 1: Linear Regression

August 20, 2023

Project 10: Build a Chatbot with LangChain and Chroma to chat with your own documents

1. Overview In this project, we will build a Retrieval-Augmented Generation Chatbot with the help of LangChain that can answer questions from internal documentation and have memory. By using Panel’s chat interface, we will also build a LangChain-powered AI chatbot for our RAG application. The Python Notebook containing the complete model development process and the data used in this project can be found at Google Drive. 2. LangChain LangChain is an open-source developer framework for building LLM applications.

August 20, 2023

Project 9: Generative QA with Retrieval-Augmented Generation (RAG) and TruEra Evaluation

1. Overview In this project, we will build a Generative Question Answering model with Retrieval-Augmented Generation (RAG) with the help of LlamaIndex that can answer questions from internal documentation. We will also evaluate, iterate, and improve the model by using TruLens. The Python Notebook containing the complete model development process and the data used in this project can be found at Google Drive. 2. Retrieval-Augmented Generation (RAG) for Question Answering (QA) In the first part of this section, we will discuss the basic RAG pipeline for generative Question Answering from internal documentation.

August 20, 2023

Project 8: Machine Translation with Transformers

1. Overview In this project, we will build a neural machine translation model with Fairseq Transformer that can translate English into Chinese naturally. The model will be trained and evaluated on the TED2020 En-Zh Bilingual Parallel Corpus. The Python Notebook containing the complete model development process and the data used in this project can be found at Google Drive. 2. Machine translation and Transformer 2.1. Brief history of machine translation The figure above illustrates the development of Machine Translation from 1950s to today (source).

August 20, 2023

Project 7: Extractive QA with a Fine-Tuned BERT

1. Overview In this project, we will build a Bidirectional Encoder Representations from Transformers (BERT) based model for a different Natural Language Processing task – Question Answering. The model will be fine-tuned on the Conversational Question Answering Challenge (CoQA) dataset from Stanford University. The Python Notebook containing the complete model development process and the data used in this project can be found at Google Drive. 2. Question Answering (QA) Question Answering, particularly Extraction-based Question Answering, is another type of Natural Language Processing task.

August 20, 2023

Project 6: Natural Language Inference with BERT and Explainable Artificial Intelligence

1. Overview In this project, we will build a Bidirectional Encoder Representations from Transformers (BERT) based model for Natural Language Inference. The performance of the model will be evaluated on the Stanford Natural Language Inference (SNLI) Corpus. To further understand how it works, we will visualize attention mechanism and compare output embedding of BERT using Euclidean distance and Cosine similarity. The Python Notebook containing the complete model development process and the data used in this project can be found at Google Drive.

August 20, 2023

Project 5: Using Autoencoder for Anomaly Detection and searching similar images

1. Overview Anomaly detection is a technique used for identifying rare items, events or observations, which deviate significantly from the majority of the data and do not conform to a well defined notion of normal behavior wikipedia. Anomaly detection has very wide applications, such as Fraud detection in credit card transactions ref, Network Intrusion detection ref, and Cancer cell detection ref. A wide spectrum of techniques have been proposed for anomaly detection, some of the popular methods are: Density-based techniques (e.

August 20, 2023

Project 4: Image Classification and Explainable Artificial Intelligence

1. Project Overview In this project, we will build a model for image classification and understand how it works. In the first part, we will develop a convolutional neural network (CNN) model for food image classification. We will also apply t-distributed Stochastic Neighbor Embedding (t-SNE) technique on the output of different layers to visualize learned visual representations of the CNN model. In order to understand how the model works, we will employ four popular Explainable AI approaches in the second part, including (1) Saliency map, (2) Smooth gradient, (3) Lime package, and (4) Integrated gradients.

August 20, 2023

Project 3: Understanding Ridesharing Demands in New York City

1. Overview In this project, we will focus on New York City’s Ridesharing Trips Data Set and try to answer three interesting questions: (1) How do the residents of New York City use Ridesharing? (2) What usage patterns can we see? (3) Can we predict usage? To answer these questions, first we will clean up the trip data and integrate them with weather data and taxi zone geodata. Then we will conduct Exploratory Data Analysis (EDA) and Statistical Tests to find Temporal and Spatial Patterns, as well as weather effects and tipping behaviors.

August 20, 2023

Project 2: Predicitng Customer Churn for a Mobile Phone Carrier

1. Overview In this project, we will build a series of models to predict the probability of Customer Churn for a Mobile Phone Carrier. In the first part, we will conduct Exploratory Data Analysis (EDA). Here the Synthetic Minority Oversampling Technique (SMOTE) method is employed to address the imbalanced classification problem. Then we will develop customer churn prediction models based on (1) Logistic Regression, (2) Decision Tree, (3) Random Forest, (4) AdaBoost, (5) Gradient Boosting Decision Trees (GBDT), and (6) Neural Network.

August 20, 2023

Project 1: Predict Real Estate Prices in Beijing

1. Overview In this project, we will build a linear regression model to help people better understand the factors that can affect the price of a house in Beijing. The main sections of this article are: (1) Exploratory Data Analysis (EDA), (2) Developing Linear Regression model for predicting real estate price, and (3) Use that model to make predictions. The Python Notebook containing the complete model development process and the data used in this project can be found at Google Drive.