Clarification Regarding the Nwords Command Line Argument - IT/Computer Science Assignment Help

Assignment Task:

Task:

Click here to access the time trials leaderboard! Changelog All changes to the assignment specification and files will be listed here. [24/06 21:20] Added clarification regarding the Nwords command-line argument. [27/06 20:10] Added more clarifications from the forum, removed the comment next to the word field in WFreq.h.

Aims

To give you practice with processing text data To give you practice implementing binary search trees To appreciate the importance of using efficient data structures and algorithms Admin Marks contributes 15% towards your final mark (see Assessment section for more details) Submit see the Submission section Deadline submit by 20:00 on Friday 16th July Late penalty 1% off the maximum mark for each hour late. For example if an assignment worth 80% was submitted 15 hours late, the late penalty would have no effect. If the same assignment was submitted 24 hours late it would be awarded 76%, the maximum mark it can achieve at that time.

Background

The field of data analytics is currently hot. Text analytics is an important subfield of data analytics. Data extracted from text documents is used in applications such as web retrieval, sentiment analysis, authorship determination, etc. In this assignment, we aim to write a program which can extract one important text analytics "measure": the frequency of occurrence of the most common words in the text. The text documents we will use are drawn from Project Gutenberg, a long-running project aimed at digitizing out-of-copyright books in simple text format and making them available for free, for anyone to use.

The books tend to be classics (such as "Moby Dick"), but are important works which Project Gutenberg aims to preserve in a simple, resilient format (ASCII text). Project Gutenberg books contain the full text of the book, but this is surrounded by meta-data and legal requirements, and it is a condition of use that these be left intact. Fortunately, the actual text of the book can be easily delineated from the other text by the following markers. ... meta-data, such as when uploaded, who provided the text, ... *** START OF THIS PROJECT GUTENBERG EBOOK title of book *** ... actual text of book ... *** END OF THIS PROJECT GUTENBERG EBOOK title of book *** ... tons of text giving licensing/legal details ... Preprocessing 6/30/2021 COMP2521 21T2 - Assignment 1 https://cgi.cse.unsw.edu.au/~cs2521/21T2/assignments/ass1 2/7 Text analysis is not as simple as carving a document into word tokens, and then using those tokens. Some additional processing is needed on the tokens before they are used in determining analytics. Three forms of processing are typically applied: tokenising/normalising English text consists of words and punctuation. We are interested primarily in the words, so we need to extract individual words from a document. We define a word as any sequence of characters that includes only alphabetics (upper and lower case), numbers, singlequote and hyphen. Once we have extracted a token, we "normalise" it by reducing to all lower-case.

This simple approach to word extraction occasionally leads to strange "words" like "'''" or "--" or "-'-". Since these kind of words occur infrequently, we allow them, and don't apply any further restrictions such as requiring at least one alphabetic character. However, we do ignore any "words" containing just a single character. stopword removal Some words are very common and make little contribution to distinguishing documents or defining the semantics of a given document, e.g., "an", "the", "you", "your", "since", etc. Such words are called "stopwords" and are typically skipped (ignored) in text analysis. We have supplied a stopword list for use in this task. stemming Words occur in different forms, e.g., "love", "loves", "lovely", "dog", "dogs", "doggy". We do not wish to distinguish such variations, and so text analysis typically reduces words to their stem.

For example, "dogs" reduces to "dog", and the forms of "love" might all reduce to "lov". We have supplied a stemming module for use in this task. The supplied stemmer is an implementation of the classic Porter stemming algorithm. It sometimes produces "unexpected" results, e.g., reducing "prince" and "princes" to "princ". This is ok; don't question the stemmer - take what the stemmer produces as The Answer.

The above  IT Assignment has been solved by our  IT Assignment  Experts at onlineassignmentbank. Our Assignment Writing Experts are efficient to provide a fresh solution to this question. We are serving more than 10000+ Students in Australia, UK & US by helping them to score HD in their academics. Our experts are well trained to follow all marking rubrics & referencing style.

Be it a used or new solution, the quality of the work submitted by our assignment experts remains unhampered. You may continue to expect the same or even better quality with the used and new assignment solution files respectively. There’s one thing to be noticed that you could choose one between the two and acquire an HD either way. You could choose a new assignment solution file to get yourself an exclusive, plagiarism (with free Turnitin file), expert quality assignment or order an old solution file that was considered worthy of the highest distinction.