How to Build a Natural Language Processing (NLP) App Assignment

Assignment Task

1. Overview

The goal of this assignment is to put together what you have learned in the course to build a user-facing end-to-end NLP application.

2. Requirements

This is the final assignment for the course, and also the most open-ended. You should perform the following steps:

1. Design and implement your NLP app;
2. Build a hosted user interface;
3. Write a user guide;
4. Present and demonstrate your system to the class More details about each step are below.

2.1. NLP App

First, you need to decide what kind of system you will build. You might think about some kind of interactive chatbot, a text tagger/classifier, translation sys- tem, multimodal system, or anything else you’ve been interested in during the course. When deciding, it is extremely important to choose something with an appropriate scope: a system that is too simple will not receive a high grade, but implementing a system that is too complex may not be doable before the deadline.

Examples of apps that would be considered too simple: Taking an existing model from huggingface, import it, and build a user interface that allows someone to modify the hyperparameters or pass inputs to that model and display the outputs. Building a front-end for a tool like ChatGPT which just routes user text to some API which does all of the processing. Exactly copying an example we have already covered in class like sentiment analysis and allowing users to analyze the sentiment of the text that they paste into a

textbox, or copying someone else’s app and just following the same steps they did from their blog post/github/etc.

Examples of apps that would be considered too complex: Collecting and manually annotating a new dataset of thousands of training examples for some task, training a model for it, and building a user interface for that model. Implementing an entirely new neural architecture for NLP that doesn’t rely on existing approaches. Training a multi-billion parameter LLM from scratch over a massive web corpus.

What is in between? Think about solving a task that people might find useful or interesting, and how you can piece together several different existing tools/approaches for doing that. Or, start with an existing tool/ap- proach/method and modify it in some useful way or fine-tune it on a new dataset in a way that allows it to do something new. Pick something you care about and that you think others will find fun/insightful/useful to interact with. The main constraint is that there should be a substantial NLP component to the app (so it should not be purely ML app working with numeric datasets, or use only image data, for example). If you are having difficulty coming up with an idea, you are always welcome to message Prof. Wilson with details on the top- ics from the class you are most interested in exploring further to help start the brainstorming process.

After you have selected your idea, think about the process: where will you get any data/models needed? How would a user interact with your system and what will a simple user interface look like? Do some exploration to see what kinds of tools/models/datasets exist already and think about how you can leverage them, even if they aren’t 100% perfect (and if there is any major limitation or existing tools, this might be where you consider some small/manageable way that you try to improve them). Start by putting together the simplest version of the system you can imagine and make sure you can transform user inputs into the correct kinds of outputs, and then build on it from there.

CSI 5180 Students must also include at least some experiments when select- ing components for the system (e.g., which pretrained LM to fine-tune, which hyperparameters to select, or even using simple approaches like bag-of-words models to compare against a transformer-based model). You should try at least 3 different approaches and evaluate them, then use this evaluation to justify why you selected the final approach that you used. Note that “evaluate” is open-ended here: you could (and probably should) include traditional metrics like f1-score, but you might also consider evaluated models based on things like inference speed (how long it takes for the model to produce outputs), RAM consumption, or a small human evaluation of the outputs. This should be doc- umented in your user guide. Note that after you have multiple approaches working, you may also choose to allow the user to select between them so they can compare the outputs themselves.

2.2. App interface

You are free to explore other tools, but Gradio is the recommended UI tool for this assignment. Gradio makes it very easy to build a demo of any kind of NLP tool, which you can host directly in Colab notebook or on Huggingface Spaces. Creating a huggingface space is not required, but is recommended because it gives a permanent “home” for your app, provides higher visibility, and you can easily show it off as part of your portfolio in the future. There are many examples of different kinds of apps already on the Gradio website which demonstrate how the interface design works, and it is fairly simple to use. This means you should be able to focus most of your time on building the underlying system that will power your app rather than tinkering with the front end. Note : if you will not have a live, hosted app, you need to have a clear justification (for example, you need to deploy your system on a specific device/hardware) and you must receive instructor approval at least 1 week in advance of the assignment deadline.

2.3. User Guide

The user guide is your main deliverable for this assignment, instead of a report like in the previous assignments. The user guide can be submitted as a PDF or written in markdown in your README file of a github repository where you store your code. It must contain the following sections:

• Introduction: an overview of what your system does. You should write this in a way that is approachable for non-technical audiences but it should still be technically accurate.
• Usage: How to run the demo, along with some example inputs/outputs that demonstrate what is cool or interesting about it. The steps to run should be very straightforward given that the model should be hosted elsewhere and run with (essentially) 1 click.
• Documentation: of models, data, and frameworks used. This should be the most detailed section of the user guide. Now you should go “behind the scenes” and explain how the demo actually works. What are the core components used? How was the data processed from user input to output (you may choose to draw a flowchart to help capture this). For pretrained models used, provide details on their architecture, training procedure, compute requirements, training data, and any know biases/limitations of the For datasets used, describe how they were collected (including how sources were selected, annotation procedures, sampling from larger subsets, filtering of dataset), the size of the dataset, distribution of classes (if applicable), etc. For external frameworks, mention which functionality was used from these frameworks. If you ran experiments (which is required for CSI 5180 students), include details about those experiments in this section along with a table of results which compares the approaches you used based on whatever evaluation metrics are appropriate to demonstrate why the selected approach was used in the end.

• Contributions: This should be a separate section that describes what you added while building your system. This does not need to be a brand new model or dataset, and can mention that you were able to link together several existing tools in a new way, that you fine-tuned a pre-trained model for a new dataset, that you chained together a series of prompts that led to a unique kind of output from a model. This section should highlight what you actually accomplished in order to build your It can cover some things you already mentioned before, but will help the instructor/TA evaluate what you did in the assignment.
• Limitations: Here you should describe what your system can’t do well (that users might reasonably expect it to do – don’t say that a sentiment analysis tool cannot also translate into French). Try to understand where the model If you have a test/evaluation set, you can directly look at examples that your best model performed pooly on and share those as well as your analysis of why that happened.

Make sure to include a link to your demo in the user guide itself.

2.4. Presentation and Demo

Instead of a final exam for this course, we will use the final exam time to have a showcase of NLP apps created by the students. Before this presentation, you should post a link to your demo in Discord in the #NLP-Apps channel and attach your user guide. You are not required to prepare any new slides (though you may choose to if helpful), but should briefly walk us through your user guide, talk about what you built and what motivated you to do that, and then show a few examples of the working demo. Each student will have around five minutes to present and demonstrate their application. You should ensure that your hosted app continues to be available for several days after class so other students can try it out.

3. Deliverables

You should submit the code you used as well as a PDF of your user guide (or github link).

3.1. Code

Your code can be written in any language but should include enough documen- tation/instructions for someone else to be able to run. You may include the code directly in your submission on Moodle (as a compressed archive) or provide a link to a GitHub repository. Your code should include a README file that explains the files/directories and how to set up and run the code.

You are welcome to use code snippets from examples in class, things you find online, or from AI code generation tools, just make sure to give proper

attribution to code you didn’t write. However, if you happen to find a codebase that does the entire assignment already (e.g., a student’s project from a past semester at OU or elsewhere), you may not just copy it and doing so will result in a grade of 0 for the assignment.

4. GCP Resources

If you need additional cloud resources (for fine-tuning models on a cloud server or using Vertex API to access Gemini or other models), Google has offered $50 of credits to each student in this course. You can use this link to activate it. However, OU admin has not set up your (or anyone’s, as far as I understand) OU account to actually use GCP. So, after you receive the activation code you should then use a different Google account to actually activate the credits (e.g., a personal Google account or a new throwaway one you just created for this purpose). If you didn’t do this, follow these steps to switch the credits to a non-OU account:

• While in your OU account, go to the Billing page
• Click on the Billing Account name (probably Billing Account for Educa- tion)
• Go to Account Management in the left menu bar Click Add Principal in the right menu bar
• Add your non-OU Gmail account under New principals
• Select Billing Account Administrator for Select a role
• Click Save
• Open GCP in your personal account and you should get a notification that you were added to a new account
• Click Accept invitation
• Proceed with the rest of the assignment under your non-OU Gmail account

5. Help/Questions

Please ask at any time on Discord or stop by office hours (in person or on Zoom) if you need advice/guidance/pointers on any aspect of the homework. You are also free to discuss your approach and ideas with your classmates, but you should not share code or reuse data. As before, you may use ChatGPT or similar AI tools if you find them helpful, but please document clearly how they were used (in your report), and remember that you take full responsibility for the deliverables that you submit.

This Engineering has been solved by our PhD Experts at My Uni Paper.