🧠 Advanced Neural Network Applications

Practical Implementations of Perceptron and Linear Neuron Models

A comprehensive educational repository demonstrating fundamental neural network architectures through practical applications including fish classification and heat influx prediction.
Features detailed mathematical analysis, step-by-step implementations, and interactive visualizations.
Perfect for learning the foundations of neural networks and machine learning.

📚 View Notebooks · 📊 Explore Datasets · 🚀 Quick Start · 📖 Documentation · 🤝 Contributing

Share Neural Network Knowledge

^{🌟 Advancing neural network education through practical implementations. Built for students, researchers, and practitioners.}

📸 Project Visualizations

[!TIP] Interactive Jupyter notebooks showcase neural network behavior through comprehensive visualizations and step-by-step analysis.

Perceptron Model - Dynamic Classification Boundary Evolution

Fish Species Classification (left) and 3D Heat Influx Prediction Surface (right)

📊 More Visualizations

Real-time Weight Update Visualization During Training

Comprehensive Error Analysis and Performance Metrics

Tech Stack Badges:

Important

This project demonstrates foundational neural network concepts through hands-on implementations. It combines theoretical understanding with practical applications, featuring perceptron models for classification and linear neurons for regression tasks. Perfect for educational purposes and research foundations.

📑 Table of Contents

🌟 Introduction

We are passionate about advancing neural network education through practical, hands-on implementations. This repository provides comprehensive educational materials demonstrating fundamental neural network architectures including perceptron models and linear neurons through real-world applications.

Whether you're a student learning neural networks, a researcher exploring foundational concepts, or a practitioner seeking implementation references, this project offers detailed mathematical analysis, step-by-step implementations, and interactive visualizations.

Note

Python 3.x required
Jupyter Notebook environment
Basic understanding of linear algebra recommended
Mathematical notation and detailed analysis included

	No installation required! View notebooks directly on GitHub.
	Comprehensive educational materials with mathematical proofs and visualizations.

Tip

⭐ Star us to support neural network education and receive updates on new implementations!

⭐ Star History

✨ Key Features

`1` Perceptron Implementation

Experience comprehensive perceptron model implementation with detailed mathematical analysis. Our approach provides step-by-step weight updates, classification boundary visualization, and convergence analysis through practical fish species classification.

Real-time Perceptron Learning with Boundary Evolution

Key capabilities include:

🎯 Binary Classification: Fish species identification (Canadian vs Alaskan)
📊 Dynamic Visualization: Real-time classification boundary updates
🔬 Mathematical Analysis: Detailed weight update calculations and proofs
📈 Performance Metrics: Error analysis and convergence tracking

`2` Linear Neuron Models

Advanced linear neuron implementations covering single-input and multi-input scenarios for heat influx prediction. Features batch learning algorithms, gradient descent optimization, and comprehensive 3D visualizations.

Single-Input Model (left) and Multi-Input Model (right) Comparison

Available Implementations:

Single-Input Model: North elevation → Heat influx prediction
Multi-Input Model: North + South elevations → Enhanced prediction accuracy
Batch Learning: Optimized training with gradient descent
3D Visualization: Interactive prediction surface rendering

`*` Educational Components

Beyond core implementations, this project includes:

📚 Comprehensive Documentation: Step-by-step mathematical derivations
🎨 Interactive Visualizations: Dynamic plots and 3D rendering capabilities
🔬 Detailed Analysis: Error metrics, R² scores, and performance comparisons
📊 Real Datasets: Fish measurement and building thermal data
🧮 Mathematical Proofs: Complete derivations of learning algorithms
💡 Educational Annotations: Extensive comments and explanations
🔄 Reproducible Results: Deterministic implementations with seed control
📱 Notebook Format: Interactive Jupyter environment for learning

✨ New educational content and implementations are continuously being added.

🛠️ Tech Stack

Python 3.x

Jupyter

NumPy

Pandas

Matplotlib

Scikit-learn

Core Technologies:

Python: Primary implementation language for neural network algorithms
Jupyter Notebooks: Interactive development and educational environment
NumPy: Efficient numerical computing and matrix operations
Pandas: Data manipulation and analysis for datasets
Matplotlib: Comprehensive plotting and visualization capabilities
Scikit-learn: Additional machine learning utilities and metrics

Mathematical Libraries:

NumPy Linear Algebra: Matrix operations and mathematical functions
SciPy: Advanced scientific computing capabilities
Mathematical Notation: LaTeX rendering in Jupyter for equations

Tip

Each technology was selected for educational clarity, mathematical precision, and visualization capabilities essential for neural network understanding.

🏗️ Architecture

Neural Network Architecture

The project implements fundamental neural network architectures with educational focus:

graph TD
    subgraph "Perceptron Model"
        A[Input Features] --> B[Linear Combination]
        B --> C[Step Activation]
        C --> D[Binary Output]
        E[Weight Updates] --> B
    end
    
    subgraph "Linear Neuron Model"
        F[Input Features] --> G[Linear Combination]
        G --> H[Linear Output]
        I[Gradient Descent] --> G
    end
    
    subgraph "Learning Process"
        J[Training Data] --> K[Forward Pass]
        K --> L[Error Calculation]
        L --> M[Backward Pass]
        M --> N[Weight Update]
        N --> K
    end

Project Structure

Educational notebook organization with progressive complexity:

advanced-neural-network-applications/
├── Part1_1.ipynb              # Perceptron: Basic Implementation
├── Part1_2.ipynb              # Perceptron: Advanced Analysis
├── Part2_1.ipynb              # Linear Neuron: Single Input
├── Part2_2.ipynb              # Linear Neuron: Optimization
├── Part2_3.ipynb              # Linear Neuron: Multi-Input
├── Part2_4.ipynb              # Linear Neuron: Validation
├── Part2_5.ipynb              # Linear Neuron: 3D Visualization
├── datasets/
│   ├── Fish_data.csv          # Fish classification dataset
│   └── heat_influx_noth_south.csv  # Building thermal data
├── docs/                      # Additional documentation
└── requirements.txt           # Python dependencies

Data Flow

sequenceDiagram
    participant D as Dataset
    participant P as Preprocessing
    participant M as Model
    participant T as Training
    participant V as Visualization
    participant A as Analysis
    
    D->>P: Load Raw Data
    P->>M: Prepared Features
    M->>T: Initialize Weights
    T->>M: Forward Pass
    M->>T: Predictions
    T->>T: Calculate Error
    T->>M: Update Weights
    M->>V: Model State
    V->>A: Visualizations
    A->>A: Performance Metrics

📚 Project Notebooks

Notebook	Topic	Description	Complexity
Part1_1.ipynb	Perceptron Basics	Binary classification with fish species data	⭐⭐
Part1_2.ipynb	Perceptron Analysis	Advanced mathematical analysis and proofs	⭐⭐⭐
Part2_1.ipynb	Linear Neuron	Single-input heat influx prediction	⭐⭐
Part2_2.ipynb	Optimization	Learning rate tuning and convergence	⭐⭐⭐
Part2_3.ipynb	Multi-Input	Two-feature regression with comparison	⭐⭐⭐
Part2_4.ipynb	Validation	Model validation and testing procedures	⭐⭐
Part2_5.ipynb	3D Visualization	Interactive 3D prediction surfaces	⭐⭐⭐⭐

Tip

Start with Part1_1.ipynb for foundational concepts, then progress through the series for comprehensive understanding.

📊 Datasets

Fish Classification Dataset

Note

Binary classification dataset for distinguishing Canadian and Alaskan fish species based on scale ring measurements.

Features: Freshwater and saltwater ring diameters
Samples: 94 fish measurements
Target: Canadian (0) vs Alaskan (1) classification
Applications: Perceptron model training and classification boundary analysis

Dataset Structure:

RingDiam_fresh_water,RingDiam_salt_water,Canadian_0_Alaskan_1
112,394,0
108,368,0
...
129,420,1

Heat Influx Dataset

Note

Regression dataset for predicting building heat influx based on elevation measurements from different directions.

Features: North and South elevation measurements
Samples: 29 building observations
Target: Heat influx values (continuous)
Applications: Linear neuron training and regression analysis

Dataset Structure:

HeatFlux,South,North
0.929,1,0.319
0.49,0.194,0.302
...

🚀 Getting Started

Prerequisites

Important

Ensure you have the following installed:

Python 3.x (Download)
Jupyter Notebook or JupyterLab
Git version control
Basic understanding of linear algebra and calculus

Quick Installation

1. Clone Repository

git clone https://github.com/ChanMeng666/advanced-neural-network-applications.git
cd advanced-neural-network-applications

2. Install Dependencies

# Using pip
pip install -r requirements.txt

# Or install manually
pip install numpy pandas matplotlib scikit-learn jupyter

3. Verify Installation

# Test imports
python -c "import numpy, pandas, matplotlib, sklearn; print('All dependencies installed successfully!')"

Launch Jupyter Notebooks

# Start Jupyter Notebook
jupyter notebook

# Or start JupyterLab (recommended)
jupyter lab

🎉 Success! Open your browser to http://localhost:8888 and navigate to the notebook files.

Recommended Learning Path:

Start with Part1_1.ipynb - Perceptron Basics
Progress to Part2_1.ipynb - Linear Neuron Introduction
Explore advanced notebooks based on your interests

📖 Implementation Details

Perceptron Model

Mathematical Foundation:

The perceptron implements binary classification using a linear decision boundary:

def perceptron_activation(inputs, weights, bias):
    """
    Perceptron activation function
    """
    return np.dot(inputs, weights) + bias

def perceptron_predict(inputs, weights, bias):
    """
    Binary classification prediction
    """
    return 1 if perceptron_activation(inputs, weights, bias) > 0 else 0

Weight Update Rule:

$$w_{new} = w_{old} + \eta \cdot (target - prediction) \cdot input$$

Where:

$\eta$ is the learning rate
$target$ is the desired output
$prediction$ is the current model output

Linear Neuron Model

Single-Input Implementation:

def linear_neuron(input_val, weight, bias):
    """
    Linear neuron for regression tasks
    """
    return weight * input_val + bias

def batch_update(inputs, targets, weight, bias, learning_rate):
    """
    Batch learning weight update
    """
    predictions = weight * inputs + bias
    errors = targets - predictions
    
    weight_update = learning_rate * np.mean(errors * inputs)
    bias_update = learning_rate * np.mean(errors)
    
    return weight + weight_update, bias + bias_update

Multi-Input Extension:

$$y = w_1 x_1 + w_2 x_2 + \ldots + w_n x_n + b$$

Mathematical Analysis

Error Metrics:

Mean Squared Error (MSE): $MSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2$
R² Score: $R^2 = 1 - \frac{SS_{res}}{SS_{tot}}$
Classification Accuracy: $Accuracy = \frac{TP + TN}{TP + TN + FP + FN}$

Tip

Each notebook includes detailed mathematical derivations and step-by-step calculations for complete understanding.

💻 Usage Examples

Fish Classification with Perceptron

Basic Usage:

import pandas as pd
import numpy as np

# Load fish classification data
data = pd.read_csv('Fish_data.csv')

# Initialize perceptron parameters
weights = np.array([102, -28])  # w1, w2
bias = 5.0
learning_rate = 0.5

# Extract training samples
first_sample = data.iloc[0]
last_sample = data.iloc[-1]

# Training loop
for epoch in range(2):
    for sample in [first_sample, last_sample]:
        x = np.array([sample['RingDiam_fresh_water'], 
                     sample['RingDiam_salt_water']])
        target = sample['Canadian_0_Alaskan_1']
        
        # Forward pass
        activation = np.dot(x, weights) + bias
        prediction = 1 if activation > 0 else 0
        
        # Weight update
        error = target - prediction
        weights += learning_rate * error * x
        bias += learning_rate * error

# Visualize classification boundary
plot_decision_boundary(weights, bias, data)

Heat Influx Prediction

Single-Input Model:

import pandas as pd
import numpy as np

# Load heat influx data
data = pd.read_csv('heat_influx_noth_south.csv')

# Initialize linear neuron
weight = -0.2
bias = 2.1
learning_rate = 0.5

# Training data
inputs = data['North'].values[:2]
targets = data['HeatFlux'].values[:2]

# Batch learning
for epoch in range(2):
    predictions = inputs * weight + bias
    errors = targets - predictions
    
    # Batch updates
    weight += learning_rate * np.mean(errors * inputs)
    bias += learning_rate * np.mean(errors)
    
    print(f"Epoch {epoch+1}: MSE = {np.mean(errors**2):.4f}")

# Make predictions
final_predictions = inputs * weight + bias

Multi-Input Enhancement:

# Multi-input linear neuron
def train_multi_input_neuron(data, epochs=3000):
    w1, w2 = np.random.randn(), np.random.randn()
    bias = np.random.randn()
    learning_rate = 0.01
    
    for epoch in range(epochs):
        # Predictions
        predictions = bias + w1 * data['North'] + w2 * data['South']
        errors = data['HeatFlux'] - predictions
        
        # Gradient descent updates
        w1 -= learning_rate * (-2 * np.mean(errors * data['North']))
        w2 -= learning_rate * (-2 * np.mean(errors * data['South']))
        bias -= learning_rate * (-2 * np.mean(errors))
        
        # Learning rate decay
        learning_rate *= 0.999
    
    return w1, w2, bias

# Train and visualize
w1, w2, bias = train_multi_input_neuron(data)
create_3d_visualization(w1, w2, bias, data)

⌨️ Development

Adding New Models

Create New Model Implementation:

# template_model.py
import numpy as np

class NewNeuralModel:
    def __init__(self, input_size, learning_rate=0.01):
        self.weights = np.random.randn(input_size)
        self.bias = np.random.randn()
        self.learning_rate = learning_rate
    
    def forward(self, inputs):
        """Implement forward pass"""
        return np.dot(inputs, self.weights) + self.bias
    
    def backward(self, inputs, targets, predictions):
        """Implement backward pass"""
        errors = targets - predictions
        self.weights += self.learning_rate * np.dot(inputs.T, errors)
        self.bias += self.learning_rate * np.mean(errors)
    
    def train(self, X, y, epochs=1000):
        """Training loop"""
        for epoch in range(epochs):
            predictions = self.forward(X)
            self.backward(X, y, predictions)
            
            if epoch % 100 == 0:
                mse = np.mean((y - predictions)**2)
                print(f"Epoch {epoch}: MSE = {mse:.4f}")

Extending Visualizations

Create Interactive Plots:

import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

def create_interactive_visualization(model, data):
    """Create comprehensive model visualization"""
    fig = plt.figure(figsize=(15, 5))
    
    # 2D Decision boundary
    ax1 = fig.add_subplot(131)
    plot_decision_boundary(model, data, ax1)
    
    # Training progress
    ax2 = fig.add_subplot(132)
    plot_training_progress(model.loss_history, ax2)
    
    # 3D Surface (for regression)
    ax3 = fig.add_subplot(133, projection='3d')
    plot_prediction_surface(model, data, ax3)
    
    plt.tight_layout()
    plt.show()

def plot_decision_boundary(model, data, ax):
    """Plot classification decision boundary"""
    # Implementation for boundary visualization
    pass

def plot_training_progress(loss_history, ax):
    """Plot training loss over time"""
    ax.plot(loss_history)
    ax.set_xlabel('Epoch')
    ax.set_ylabel('Loss')
    ax.set_title('Training Progress')

🧪 Testing

Run Comprehensive Tests:

# Test all notebook implementations
python -m pytest tests/

# Test specific models
python tests/test_perceptron.py
python tests/test_linear_neuron.py

# Verify mathematical calculations
python tests/test_mathematical_accuracy.py

Validation Framework:

def validate_implementation(model, expected_results):
    """Validate model implementation against known results"""
    tolerance = 1e-6
    
    for test_case in expected_results:
        inputs = test_case['inputs']
        expected = test_case['expected']
        actual = model.predict(inputs)
        
        assert abs(actual - expected) < tolerance, \
            f"Expected {expected}, got {actual}"
    
    print("All validation tests passed!")

# Mathematical accuracy tests
def test_weight_updates():
    """Test perceptron weight update calculations"""
    # Known test cases with hand-calculated results
    test_cases = [
        {
            'inputs': np.array([112, 394]),
            'target': 0,
            'initial_weights': np.array([102, -28]),
            'initial_bias': 5.0,
            'learning_rate': 0.5,
            'expected_weights': np.array([46, -225]),
            'expected_bias': 4.5
        }
    ]
    
    for case in test_cases:
        # Implement test logic
        pass

🤝 Contributing

We welcome contributions to advance neural network education! Here's how you can help improve this project:

Development Process

1. Fork & Clone:

git clone https://github.com/ChanMeng666/advanced-neural-network-applications.git
cd advanced-neural-network-applications

2. Create Feature Branch:

git checkout -b feature/new-neural-model

3. Development Setup:

# Install development dependencies
pip install -r requirements-dev.txt

# Install Jupyter extensions
jupyter contrib nbextension install --user
jupyter nbextension enable --py widgetsnbextension

4. Code Guidelines:

✅ Follow PEP 8 style guidelines
✅ Include comprehensive mathematical documentation
✅ Add educational comments and explanations
✅ Provide step-by-step implementation details
✅ Include visualization and analysis components

5. Submit Pull Request:

Provide clear description of educational value
Include mathematical proofs and derivations
Reference related academic concepts
Ensure all notebooks run without errors

Contribution Guidelines

Educational Content:

Clear mathematical explanations with LaTeX formatting
Step-by-step implementation walkthroughs
Comprehensive visualizations and plots
Real-world application examples
Interactive elements for learning

Code Quality:

Well-documented functions with docstrings
Consistent naming conventions
Modular and reusable implementations
Error handling and input validation
Performance optimization where appropriate

Issue Reporting:

🐛 Bug Reports: Include notebook and cell information
💡 Feature Requests: Suggest new neural network implementations
📚 Documentation: Help improve mathematical explanations
🎓 Educational Ideas: Propose new learning scenarios

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

Open Source Benefits:

✅ Commercial use allowed
✅ Modification allowed
✅ Distribution allowed
✅ Private use allowed
✅ Educational use encouraged

👥 Team

_{Chan Meng}
Creator & Lead Developer
Neural Network Researcher

🙋‍♀️ Author

Chan Meng - Neural Network & Machine Learning Engineer

LinkedIn: chanmeng666
GitHub: ChanMeng666
Email: [email protected]
Website: chanmeng.live

Research Interests:

🧠 Neural Network Architectures: Fundamental and advanced implementations
📊 Machine Learning: Supervised and unsupervised learning algorithms
🎓 Educational Technology: Interactive learning tools and visualizations
🔬 Applied Mathematics: Mathematical foundations of AI and ML

🧠 Advancing Neural Network Education Through Practice 📚
Empowering students and researchers with hands-on neural network implementations

⭐ Star us on GitHub • 📖 Explore the Notebooks • 🐛 Report Issues • 💡 Suggest Features • 🤝 Contribute

Made with ❤️ for the machine learning community

Name		Name	Last commit message	Last commit date
Latest commit History 31 Commits
.github		.github
.gitattributes		.gitattributes
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
Fish_data.csv		Fish_data.csv
LICENSE		LICENSE
Part1_1 copy.ipynb		Part1_1 copy.ipynb
Part1_1.ipynb		Part1_1.ipynb
Part1_2 copy.ipynb		Part1_2 copy.ipynb
Part1_2.ipynb		Part1_2.ipynb
Part2_1 copy.ipynb		Part2_1 copy.ipynb
Part2_1.ipynb		Part2_1.ipynb
Part2_2 copy.ipynb		Part2_2 copy.ipynb
Part2_2.ipynb		Part2_2.ipynb
Part2_3 copy.ipynb		Part2_3 copy.ipynb
Part2_3.ipynb		Part2_3.ipynb
Part2_4.ipynb		Part2_4.ipynb
Part2_5 copy.ipynb		Part2_5 copy.ipynb
Part2_5.ipynb		Part2_5.ipynb
README.md		README.md
heat_influx_noth_south.csv		heat_influx_noth_south.csv

Uh oh!

License

ChanMeng666/advanced-neural-network-applications

Folders and files

Latest commit

History

Repository files navigation

🧠 Advanced Neural Network Applications

Practical Implementations of Perceptron and Linear Neuron Models

📸 Project Visualizations

TOC

🌟 Introduction

✨ Key Features

1 Perceptron Implementation

2 Linear Neuron Models

* Educational Components

🛠️ Tech Stack

🏗️ Architecture

Neural Network Architecture

Project Structure

Data Flow

📚 Project Notebooks

📊 Datasets

Fish Classification Dataset

Heat Influx Dataset

🚀 Getting Started

Prerequisites

Quick Installation

Launch Jupyter Notebooks

📖 Implementation Details

Perceptron Model

Linear Neuron Model

Mathematical Analysis

💻 Usage Examples

Fish Classification with Perceptron

Heat Influx Prediction

⌨️ Development

Adding New Models

Extending Visualizations

🧪 Testing

🤝 Contributing

Development Process

Contribution Guidelines

📄 License

👥 Team

🙋‍♀️ Author

About

Topics

Resources

License

Code of conduct

Uh oh!

Stars

Watchers

Forks

Releases

Sponsor this project

Uh oh!

Packages 0

Languages

`1` Perceptron Implementation

`2` Linear Neuron Models

`*` Educational Components

Packages