Below you will find pages that utilize the taxonomy term “Natural Language Processing”
Post
Project 10: Build a Chatbot with LangChain and Chroma to chat with your own documents
1. Overview In this project, we will build a Retrieval-Augmented Generation Chatbot with the help of LangChain that can answer questions from internal documentation and have memory. By using Panel’s chat interface, we will also build a LangChain-powered AI chatbot for our RAG application.
The Python Notebook containing the complete model development process and the data used in this project can be found at Google Drive.
2. LangChain LangChain is an open-source developer framework for building LLM applications.
Post
Project 9: Generative QA with Retrieval-Augmented Generation (RAG) and TruEra Evaluation
1. Overview In this project, we will build a Generative Question Answering model with Retrieval-Augmented Generation (RAG) with the help of LlamaIndex that can answer questions from internal documentation. We will also evaluate, iterate, and improve the model by using TruLens.
The Python Notebook containing the complete model development process and the data used in this project can be found at Google Drive.
2. Retrieval-Augmented Generation (RAG) for Question Answering (QA) In the first part of this section, we will discuss the basic RAG pipeline for generative Question Answering from internal documentation.
Post
Project 8: Machine Translation with Transformers
1. Overview In this project, we will build a neural machine translation model with Fairseq Transformer that can translate English into Chinese naturally. The model will be trained and evaluated on the TED2020 En-Zh Bilingual Parallel Corpus.
The Python Notebook containing the complete model development process and the data used in this project can be found at Google Drive.
2. Machine translation and Transformer 2.1. Brief history of machine translation The figure above illustrates the development of Machine Translation from 1950s to today (source).
Post
Project 7: Extractive QA with a Fine-Tuned BERT
1. Overview In this project, we will build a Bidirectional Encoder Representations from Transformers (BERT) based model for a different Natural Language Processing task – Question Answering. The model will be fine-tuned on the Conversational Question Answering Challenge (CoQA) dataset from Stanford University.
The Python Notebook containing the complete model development process and the data used in this project can be found at Google Drive.
2. Question Answering (QA) Question Answering, particularly Extraction-based Question Answering, is another type of Natural Language Processing task.
Post
Project 6: Natural Language Inference with BERT and Explainable Artificial Intelligence
1. Overview In this project, we will build a Bidirectional Encoder Representations from Transformers (BERT) based model for Natural Language Inference. The performance of the model will be evaluated on the Stanford Natural Language Inference (SNLI) Corpus. To further understand how it works, we will visualize attention mechanism and compare output embedding of BERT using Euclidean distance and Cosine similarity.
The Python Notebook containing the complete model development process and the data used in this project can be found at Google Drive.