Transformer Thermal Anomaly Detection System

Project Overview

The Transformer Thermal Anomaly Detection System is a web-based application designed for the inspection of electrical transformers using thermal imaging. It leverages an AI-powered backend to analyze thermal images against a baseline, automatically detecting and classifying potential faults like loose joints and overloads.

This is a collaborative team project developed as part of the EN3350 Software Design Competition at the University of Moratuwa. The project addresses a real-world challenge in power distribution network maintenance, where electric utilities routinely perform thermal imaging to uncover early warning signs such as overheating, loose joints, insulation degradation, and load imbalance.

The Challenge

Traditional thermal inspection workflows rely on manual side-by-side visual comparison, which is slow, subjective, and difficult to audit. Our solution digitizes this process with:

• Automated transformer master data and baseline/maintenance image management
• AI-powered anomaly detection comparing new thermal images against stored baselines
• Interactive annotation tools enabling engineers to validate and augment AI findings (human-in-the-loop)
• Structured, printable maintenance record sheets with annotated imagery for traceable operational decisions

Project Objectives

The goal is to create a traceable, efficient, and extensible inspection pipeline that reduces human error, accelerates decision-making, and builds a reliable historical knowledge base for asset management. The system was designed to meet phase-wise functional requirements with emphasis on scalability, efficiency, ML integration, and code quality.

Key Features

Technology Stack

Frontend

• React.js: Component-based UI framework for building the interactive inspection interface
• Canvas API: Interactive image annotation and bounding box drawing functionality
• Axios: HTTP client for communicating with the Flask backend API
• CSS3: Responsive styling with design tokens and theming via CSS variables
• Reusable Components: Modal systems, zoom viewers, annotation tools, and form inputs

Backend

• Python 3.10+: Core programming language for backend logic
• Flask: Lightweight web framework for RESTful API endpoints
• Flask-CORS: Cross-Origin Resource Sharing for frontend-backend communication
• SQLite: Embedded relational database for persistent data storage
• ReportLab: PDF generation library for maintenance record export

Computer Vision & AI

• OpenCV (opencv-python): Image processing, alignment, and transformation
• NumPy: Numerical computations for image array manipulation
• scikit-image: Color difference analysis and thermal anomaly detection algorithms
• Classical CV Pipeline: Feature-based alignment, threshold segmentation, morphological filtering
• Heuristic Classification: Severity scoring and subtype detection (LooseJoint, PointOverload)

Project Structure

• backend/: Flask API (app.py), CV detection logic (anomaly_cv.py), database schema & migrations
• core4/: React frontend with modular components, themed styling, and state management
• data/: Sample transformer thermal image datasets for testing
• Modular Design: Clear separation of concerns for maintainability and testing

System Architecture

The system follows a client-server architecture with a React frontend and Python Flask backend:

Frontend Layer (React)

• Single-page application running on http://localhost:3000
• Image upload interface for baseline and thermal images
• Interactive canvas for drawing and editing bounding boxes
• Analysis log table displaying all annotation actions with timestamps
• Classification dropdowns (Faulty, Potentially Faulty, Normal) for manual annotations
• Export functionality for JSON and CSV annotation logs

Backend Layer (Flask)

• REST API running on http://localhost:8000
• Endpoints for saving/loading annotations: /api/annotations/<inspection_id>
• SQLite database management (transformers, inspections, annotations, annotation_logs)
• Image processing service using OpenCV and scikit-image
• AI inference for automatic anomaly detection and classification

Database Schema

• transformers: Transformer ID, number, location, baseline images
• inspections: Inspection ID, transformer reference, thermal images, timestamps
• annotations: Bounding boxes with coordinates, created_at, updated_at, user_id, notes, classification
• annotation_logs: Complete audit trail of all add/edit/delete actions

AI/CV Processing Pipeline

• Image Alignment: Automatic registration of baseline and thermal images
• Color Difference Analysis: Pixel-level thermal variation detection
• Hotspot Detection: Identification of temperature anomalies
• Classification: Automatic categorization (LooseJoint, PointOverload, etc.)
• Bounding Box Generation: Automated marking of detected anomalies

Annotation System

The annotation system allows users to interactively add, edit, and delete bounding boxes (markers) on thermal images to identify anomalies. Each annotation records:

• created_at: Timestamp when the bounding box was first added (set by frontend, preserved by backend)
• updated_at: Timestamp of the last edit or resize (set by frontend, preserved by backend)
• user_id: The user who performed the action
• notes: Optional comments or reasons for the annotation
• classification: Faulty, Potentially Faulty, or Normal

Annotations are displayed in the Analysis Log table, showing the user and time for each action. The log persists across saves and reloads, accurately reflecting when each bounding box was added or edited.

Backend Annotation Persistence

• Database Table: Annotations stored in the annotations table with all metadata fields
• API Endpoint: /api/annotations/<inspection_id> for saving and loading annotations
• Persistence Logic: Backend checks for created_at and updated_at fields from client and stores as provided

Export Formats

The system provides export functionality for annotation logs in both JSON and CSV formats:

• JSON Export: Grouped by inspection, each containing transformer, images, and annotation actions
• CSV Export: Flat table with rows for each annotation action, clear columns for inspection, transformer, image, details
• De-duplication: Only first occurrence (earliest timestamp) of each action type (added/edited/deleted) included
• Human-readable IDs: Inspection numbers like T1-INSP1, clear transformer and image identifiers

Phase-wise Development

The project was developed in four structured phases aligned with the EN3350 competition requirements. Each phase built upon the previous, creating a comprehensive end-to-end inspection system:

Phase 1: Transformer & Baseline Management

• Transformer CRUD: Create, list, edit, delete transformer records with ID, location, capacity
• Image Upload: Associate baseline and maintenance images with transformers
• Environment Tagging: Label baseline images (Sunny/Cloudy/Rainy) for contextual comparison

Phase 2: Automated Anomaly Detection

• Comparison Engine: Align images, compute thermal deltas, identify hotspot candidates
• Visual Side-by-Side: Synchronized baseline and maintenance image viewing
• Automatic Marking: Bounding boxes with severity heuristics and subtype classification

Phase 3: Interactive Annotation & Feedback

• Editing Tools: Resize, move, delete boxes; add new manual anomalies
• Metadata Persistence: Timestamps, classification override, notes, source tracking (AI/User)
• Feedback Log: Historical action trace for future model improvement

Phase 4: Maintenance Record Generation

• Record Form: Pre-fills transformer metadata, inspection number, annotated image snapshot
• Engineer Inputs: Status, voltage/current readings, recommended action, remarks
• Persistence & Export: Snapshot-based records with PDF export capability

Project Completion Status

All four phases have been successfully implemented and tested. Here's the final status:

Project Status: Successfully completed for EN3350 Software Design Competition (November 2025). All four phases implemented with full functionality, comprehensive testing, and documentation. The system demonstrates effective integration of classical computer vision, interactive UI, and structured data management for real-world transformer inspection workflows.

Known Limitations & Future Improvements

Setup & Installation

The project requires setting up both the backend (Flask) and frontend (React) components:

Prerequisites

• Node.js and npm: Required for running the React frontend
• Python 3.x and pip: Required for running the Flask backend

Backend Setup

• Navigate to backend/ directory
• Create Python virtual environment: python -m venv venv
• Activate virtual environment (Windows: .\venv\Scripts\activate, macOS/Linux: source venv/bin/activate)
• Install dependencies: pip install Flask Flask-Cors numpy opencv-python scikit-image
• Initialize database: python database.py (creates backend.db)
• Run server: python app.py (runs on http://localhost:8000)

Frontend Setup

• Open a new terminal window
• Navigate to core4/ directory
• Install dependencies: npm install
• Run development server: npm start (runs on http://localhost:3000)

Database Management

• Clear all data: From backend directory, run the database clearing command to reset all tables
• Note: All data will be erased from the database when clearing

Note: Detailed setup instructions and troubleshooting guide are available in the project repository README.

My Contributions

This project was developed as a collaborative team effort, with each member contributing their expertise to different aspects of the system. My primary contributions focused on the frontend development and system integration phases.

Phase 1: Complete Frontend Development

• Initial Web Application: Developed the complete webpage interface with all visual components and working logic as the foundation of the project
• Interactive UI: Implemented the image upload interface, annotation canvas, and analysis log display
• Frontend Logic: Created the initial JavaScript logic for bounding box drawing, editing, and deletion functionality
• User Workflow: Designed the inspection workflow from image upload to report generation

Phase 2: Code Modularization & Refinement

• Team Collaboration: Worked with team members who modularized and refactored the initial codebase for better maintainability
• Code Review: Participated in code reviews and integration testing of modularized components

Phase 3: OpenCV Integration & Final Setup

• System Integration: Developed the final working setup integrating OpenCV for image recognition and thermal analysis
• Parallel Development: Worked alongside team members in parallel development due to time constraints, comparing different approaches to select the best implementation
• Testing & Optimization: Conducted extensive testing of the integrated system to ensure smooth communication between frontend and backend
• ML Integration: Collaborated with the ML-focused team member who developed the OpenCV computer vision algorithms for anomaly detection

Team Collaboration: This project showcases effective teamwork, with the ML specialist focusing on the computer vision algorithms while I concentrated on the web application development and system integration. The parallel development approach allowed us to explore multiple solutions and deliver a robust final product within the project timeline.

Learning Outcomes

• Full-stack web development with React and Flask
• Computer vision techniques using OpenCV (image alignment, color analysis)
• AI-based anomaly detection and classification systems
• RESTful API design and client-server architecture
• SQLite database design and data persistence
• Interactive canvas-based annotation tools
• Team collaboration on software development projects with parallel development workflows
• Managing development vs. production environments
• Practical application of computer vision in industrial systems
• User interface design for technical inspection workflows
• Integration of frontend applications with ML/AI backend services