UEL-CN-7031: Big Data Analytics Assessment

This coursework (CRWK) must be completed as an individual effort. The assessment is divided into two main sections:

1. Big Data Analytics Report – approximately 5,000 words (±10%) – 60% of total grade

2. Presentation – approximately 1,000 words (±10%) – 40% of total grade

Marking Scheme Big Data Analytics Report

The report evaluates multiple tasks based on technical execution, analytics, and documentation. The assessment areas include:

Big Data Analytics Using Hive (30 marks)

• Hive Queries (10 marks): Students must provide multiple complex queries using Apache Hive to extract meaningful information from the dataset. Basic SELECT queries are insufficient to achieve full marks.

• Built-in Functions (10 marks): Correctly apply Hive built-in functions such as Date, Math, Conditional, and String functions to enhance data querying.

• Visualization & Interpretation (10 marks): Visualize the results of Hive queries using appropriate tools and provide a clear, concise interpretation of findings.

Big Data Analytics Using Spark (50 marks)

• Statistical Analysis (15 marks): Conduct descriptive and inferential analysis on the dataset using PySpark. Include methods such as correlation, hypothesis testing, and density estimation. Present numerical and graphical results with clear labeling, legends, and tooltips for clarity.

• Design and Build Classifiers (35 marks):

  • Binary Classifier (15 marks): Build and explain a binary classification model. Evaluate its accuracy and performance, presenting results both numerically and graphically.

  • Multi-class Classifier (20 marks): Develop a classifier to categorize data into ten classes – one normal and nine attack types (Fuzzers, Analysis, Backdoors, DoS, Exploits, Generic, Reconnaissance, Shellcode, Worms). Explain model parameters, accuracy, and effectiveness.

Individual Assessment (10 marks)

• Discuss alternative technologies or analytics approaches for Hive and Spark tasks, referencing academic sources.

• Reflect on new insights gained during the analytics process and areas that were challenging or unexpected.

Documentation (10 marks)

• Present work in a structured, scientific report format with clarity, academic tone, and proper referencing.

Dataset Overview

The coursework uses the UNSW-NB15 dataset, generated to simulate real network activity alongside synthetic attack behavior. Key points include:

• Captured using the Tcpdump tool in a Cyber Range Lab.

• Includes nine types of attacks: Fuzzers, Analysis, Backdoors, DoS, Exploits, Generic, Reconnaissance, Shellcode, and Worms.

• Argus and Bro-IDS tools were used to extract 49 features with class labels.

• The coursework utilizes 10 million records stored in a CSV file (~600MB), suitable for big data analytics using Hive and Spark.

Students should first explore the dataset, import it into Hadoop HDFS, and query a few records using Hive to understand feature distributions.

Big Data Query & Analysis Using Hive

Hive converts large raw datasets into structured, meaningful information. Students are expected to:

1. Understand the dataset thoroughly before querying.

2. Design and implement at least four complex Hive queries that go beyond simple selection.

3. Apply Hive functions (Date, Math, Conditional, String) to enhance query results.

4. Visualize query outputs numerically and graphically.

5. Interpret findings clearly in the report.

6. Include screenshots of queries and results in the report.

Key Objective: Develop practical skills in querying and interpreting large datasets, highlighting meaningful patterns and insights.

Advanced Analytics Using PySpark

Advanced analytics is conducted using PySpark to extract deeper insights and build predictive models. This task includes:

Analyzing and Interpreting Big Data (15 marks)

• Apply at least four statistical analysis methods, such as descriptive statistics, correlation analysis, hypothesis testing, and density estimation.

• Present results in both tabular and graphical formats with proper labels, legends, and tooltips to support understanding.

• Interpret results for decision-making and insight generation.

Design and Build Classifiers (35 marks)

• Binary Classification (15 marks): Build a model to differentiate between normal and attack traffic. Explain algorithm choice, configuration, and performance metrics.

• Multi-class Classification (20 marks): Classify data into ten categories, including nine attack types. Include an explanation of model design, parameters, accuracy, and effectiveness.

Key Objective: Demonstrate the ability to apply machine learning techniques for large-scale data, evaluate model performance, and visualize results for clear communication.

Task 3: Individual Assessment (10 marks)

Students must reflect individually on:

• Alternative solutions: Other tools and technologies for performing similar tasks in Hive and Spark, including differences in efficiency, usability, and scalability.

• Learning insights: Personal reflections on new concepts learned, unexpected findings, and knowledge gaps discovered during the analysis.

Key Objective: Encourage critical thinking and personal reflection on methodologies, technologies, and big data analytics approaches.

Task 4: Documentation (10 marks)

The final report must:

• Include a cover page and table of contents.

• Organize tasks into clear subsections.

• Include proper references using academic sources.

• Demonstrate academic writing quality, integrity, and coherence.

• Be submitted as a single PDF file through Turnitin.

Key Objective: Develop professional reporting skills suitable for academic and industry standards.

Presentation Requirements 

The presentation is based on the analytics report and should follow these guidelines:

• Content (50 marks): Cover the topic in-depth with clear explanation and supporting evidence.

• Design & Layout (20 marks): Use professional slides, including graphics, fonts, colors, effects, and transitions to enhance understanding.

• Length (10 marks): Ensure the presentation aligns with the word count requirement (approx. 1,000 words).

• Organization (20 marks): Present information logically, with a coherent flow for the audience.

Key Objective: Develop clear communication and visualization skills to convey complex analytics to an audience effectively.

Submission Guidelines

• Submit a single PDF file for the report and another single PDF for the presentation.

• Use Turnitin submission portals by the end of Week 12.

• Ensure the final submission adheres to the provided structure: cover page, table of contents, report sections, references, and presentation.

Plagiarism Policy

The University strictly prohibits submission of material produced by another person without due acknowledgment. Examples include:

• Using written, visual, or oral material from others without proper referencing.

• Incorporating significant extracts from others’ work.

• Collaborative work submitted as individual assessment.

All forms of plagiarism or collusion are considered assessment offences. Students must ensure their work is original and properly cited.

Conclusion

The UEL-CN-7031 Big Data Analytics coursework provides students with an opportunity to develop skills in big data querying, analysis, visualization, and predictive modeling using Hive and Spark. Through this assessment, students will:

• Gain practical experience in managing large datasets (10 million records).

• Apply analytical and statistical methods for meaningful insights.

• Build and evaluate binary and multi-class classifiers.

• Reflect on alternative solutions and personal learning.

• Present findings professionally in both written and visual formats.

Summary of Assessment Requirements

The UEL-CN-7031 Big Data Analytics coursework is an individual assessment divided into two main components:

1. Big Data Analytics Report – approximately 5,000 words (±10%), contributing 60% of the final grade.

2. Presentation – approximately 1,000 words (±10%), contributing 40% of the final grade.

Key Requirements for the Report:

• Big Data Analytics Using Hive (30 marks):

  • Develop at least four complex Hive queries that extract meaningful insights.

  • Use built-in Hive functions (Date, Math, Conditional, String) effectively.

  • Visualize query outputs and provide clear interpretations.

• Big Data Analytics Using Spark (50 marks):

  • Conduct statistical analyses such as descriptive statistics, correlation, hypothesis testing, and density estimation.

  • Build a binary classifier to distinguish between normal and attack traffic.

  • Develop a multi-class classifier to categorize data into ten classes: one normal and nine attack types (Fuzzers, Analysis, Backdoors, DoS, Exploits, Generic, Reconnaissance, Shellcode, Worms).

  • Evaluate model accuracy and effectiveness, presenting results both numerically and graphically.

• Individual Assessment (10 marks):

  • Discuss alternative technologies or approaches for Hive and Spark analytics.

  • Reflect on personal learning, challenges faced, and new insights gained.

• Documentation (10 marks):

  • Organize work in a structured scientific report.

  • Include a cover page, table of contents, references, and clearly labeled sections.

  • Ensure academic writing quality, coherence, and adherence to referencing standards.

Dataset Overview:

• The UNSW-NB15 dataset simulates real network activity combined with synthetic cyber-attack behaviors.

• Includes nine types of attacks and 49 features extracted using Argus and Bro-IDS tools.

• The dataset contains 10 million records (~600MB), making it ideal for big data analytics using Hive and Spark.

Presentation Requirements (40% of final grade):

• Present the key findings of the report in approximately 1,000 words.

• Ensure logical sequence, professional design, effective use of graphics, and clear communication of results.

Approach Adopted by Academic Mentor

The academic mentor guided the student through a structured, step-by-step process to ensure comprehensive completion of the assessment while meeting all learning objectives.

Step 1: Understanding Dataset and Project Scope

• The mentor began by explaining the UNSW-NB15 dataset and its structure.

• Students were advised to import a sample of the data into Hadoop HDFS and perform initial Hive queries to explore feature distributions.

• The mentor highlighted the importance of understanding attack types, features, and dataset volume to plan analysis.

Learning Objective Covered: Develop foundational understanding of big data and dataset features.

Step 2: Big Data Querying Using Hive

• The mentor guided the student to design four complex Hive queries rather than simple SELECT statements.

• Instructions included the use of built-in Hive functions to manipulate and analyze data (Date, Math, Conditional, String).

• Students were taught how to visualize the query results using appropriate graphical tools and interpret findings effectively.

Learning Objective Covered: Gain practical skills in querying and visualizing large datasets.

Step 3: Advanced Analytics Using PySpark

• The mentor demonstrated how to conduct descriptive and inferential statistical analyses (correlation, hypothesis testing, density estimation).

• Students were guided to build a binary classifier to separate normal traffic from attacks and a multi-class classifier for ten categories.

• Mentor emphasized the importance of documenting model configuration, performance metrics, and graphical outputs for clarity.

Learning Objective Covered: Apply machine learning for large-scale datasets, evaluate model performance, and present results clearly.

Step 4: Individual Assessment and Reflection

• Students were instructed to research alternative technologies and analytics approaches for Hive and Spark tasks.

• Mentor encouraged reflection on personal learning, including challenges, new insights, and areas needing improvement.

Learning Objective Covered: Develop critical thinking, reflective practice, and understanding of alternative analytics approaches.

Step 5: Documentation and Report Compilation

• The mentor guided students in structuring the report with a cover page, table of contents, clear subsections, and references.

• Emphasis was placed on maintaining academic integrity and coherence, ensuring adherence to submission guidelines and Turnitin requirements.

Learning Objective Covered: Produce professional, structured, and academically sound reports suitable for academic and industry standards.

Step 6: Presentation Preparation

• Mentor provided tips for effective communication of findings through a 1,000-word presentation.

• Students were instructed to use logical sequencing, professional slide design, graphics, and clear labeling to communicate complex analytics insights.

Learning Objective Covered: Develop effective presentation and visualization skills for conveying technical information.

Outcome Achieved

• Students successfully completed the Hive queries, statistical analyses, and classifier modeling.

• Full documentation and presentation were produced, adhering to university guidelines.

• Students gained a thorough understanding of big data analytics, machine learning techniques, and reflective assessment practices.

• Key learning objectives, including dataset exploration, data visualization, classifier building, critical reflection, and professional reporting, were achieved.

Boost Your Academic Success with Reliable Assignment Solutions

Looking to get a clear idea of how to approach your assignment? You can download our sample solution to understand the structure, methodology, and key concepts. Remember: this sample is for reference only—submitting it as your own work may result in plagiarism and academic penalties.

For a completely original, plagiarism-free solution, our team of professional academic writers can create a custom assignment tailored to your requirements. Ordering a fresh solution ensures:

• 100% Unique Content: Written from scratch to meet your specific assignment criteria.

• Expert Guidance: Developed by qualified professionals with subject expertise.

• High-Quality Presentation: Proper formatting, referencing, and academic tone.

• On-Time Delivery: Receive your assignment ready to submit without stress.

Protect your grades and your academic integrity by using the sample for guidance only, and trust our experts to provide a fresh, top-quality solution that supports your learning.