a14g-final-submission-s25-t21-pixel-circuit

Review Assignment Due Date

a14g-final-submission

* Team Number: 21
* Team Name: pixel & circuit
* Team Members: Zhongyu Wang, Linhai Deng
* Github Repository URL: https://github.com/ese5160/a14g-final-submission-s25-t21-pixel-circuit.git
* Description of test hardware: (development boards, sensors, actuators, laptop + OS, etc) laptop

1. Video Presentation

YouTube Shorts Video

2. Project Summary

Device Description

Device Functionality

Element Details
Sensors Murata IRA‑S210ST01 PIR → ADC PA03. Threshold 1000 raw (≈1.2 V) → attention vs safe.
Actuators I²C OLED (UG‑2864HSWEG01) + on‑board LED. Panic button toggles text “the police is coming” and LED.
Speaker footprint present but disabled (MCU RAM limits).
Live video ESP‑EYE 2 MP camera streams MJPEG + built‑in mic (no extra MCU RAM).
Snapshot & mail Each attention sends JPEG + timestamp to user‑supplied e‑mail via Node‑RED SMTP.
OTA update Dashboard buttons GOLDEN IMAGE / OTA UPDATE (FW) push a firmware URL; board downloads and self‑flashes.

1.2

 Challenges

# Issue Mitigation / Result
1 Tight MCU RAM – 14 kB total, 11 kB pre‑used by vendor Wi‑Fi stack • Moved OLED font table to flash (const PROGMEM)
• Disabled WAV codec & speaker driver to free 1.7 kB
• Reduced FreeRTOS task stacks by 20 %
2 No PWM on PA02 (speaker pin) • Generated square wave via 10‑bit DAC as stop‑gap buzzer
• Added “speaker → TCC1 WO[0]” note to PCB‑v2 issues list
3 ISR ↔ MQTT race conditions • Created two FreeRTOS queues xQueueDoorEvents & xQueueBuzzerCmd so ISRs only enqueue, never touch sockets
• MQTT publish handled in Wi‑Fi task context
4 Low‑budget PIR latency • $2 Murata PIR selected for BoM limit → 3 m range & 900 ms rise‑time
• Debounce filter added, but long latency remains (see Future Work)
5 Node-RED not optimized for video • Switched from real video to periodic JPEG frames (~1–3 fps)
• Frame sent as Base64 via MQTT or HTTP POST, displayed with <img> in dashboard
6 Web dashboard frame flicker • Introduced timestamp-based buffer to avoid image overwrite conflicts
• Only update display if new frame differs from previous
7 Large JPEG frames cause MQTT choke • Capped resolution to QQVGA (160×120), frame <5 kB
• Allowed only 1 in-flight image at a time to prevent overload
8 Browser-side image memory growth • Manually revoked <img> blob URLs and cleared DOM nodes every 10 frames
• Exploring <canvas>-based rendering in Future Work

 Prototype Learnings

 Next Steps & Takeaways

Next Steps

  1. PCB v2
    • Move speaker to a true PWM pin (TCC1/WO[0])
    • Add 2 MB SPI flash for snapshot buffering
    • Integrate Li‑ion fuel gauge & charge‑status LED
  2. Full‑duplex audio
    • Stream ESP‑EYE microphone to dashboard (WebRTC)
    • Relay doorbell chime + pre‑recorded messages back to speaker
  3. Ultra‑low‑power mode
    • Target < 200 µA standby with PIR + WINC1500 M2M_wake interrupt
    • Periodic RTC wake to push “heartbeat” MQTT packet
  4. PIR upgrade
    • Replace analog Murata IRA‑S210ST01 with digital Panasonic EKMC series (6 m range, 170 ms latency)
    • Remove 100 kΩ pull‑down and averaging filter once latency improves
  5. Web-based video frontend
    • Replace Node-RED with Flask + WebSocket server for smoother JPEG streaming
    • Use <canvas> or <video> HTML5 elements for improved rendering control
    • Support frame annotations (bounding boxes, timestamps)
  6. MJPEG / HLS streaming
    • Explore MJPEG over HTTP or HLS (HTTP Live Streaming) for better compatibility
    • Reduce latency and improve real-time responsiveness of video feed
  7. Video-aware flow control
    • Implement adaptive frame rate based on network condition and browser feedback
    • Buffer overflow protection and dropped-frame detection via frame index tracking

Course Takeaways

Node-RED instance: http://20.55.16.155:1880/ui

Altium 365: https://upenn-eselabs.365.altium.com/designs/AF1AAD73-CCAB-4D4B-BED5-52957B402218?activeDocumentId=Top20Level.SchDoc(1)&variant=%5BNo+Variations%5D&activeView=SCH&location=%5B1,97.61,18.77,22.79%5D#design

3. Hardware & Software Requirements

Req ID Requirement (from design spec) Target Metric Test / Validation Method Result Notes
HW‑01 PIR motion detection latency / range < 1 s from movement to IRQ Oscilloscope on PIR AIN vs. UART “attention” print‑out ⚠️ Budget PIR; see cost‑driven challenge note
HW‑02 ADC noise floor (12‑bit SAR) ±3 LSB max @ 1 kHz Logged 1 000 samples, calculated σ ✅  3.3 V reference, RC filter
HW‑03 OLED legibility indoors 700 cd/m² min Lux‑meter on white screen ✅ Contrast 100 %
HW‑04 Battery life (Li‑Ion 800 mAh) ≥ 24 h standby Simulated with bench supply @ 30 µA sleep ⚠️ Needs deep‑sleep optimisation
HW‑05 ESP32-S3-EYE JPEG frame capture ≥ 1 fps @ QQVGA (160×120) Measured frame interval with timestamped logs ✅ 2.3 fps avg; JPEG size ~4.6 kB
SW‑01 FreeRTOS task scheduling jitter (sensor task) < 5 ms vTaskGetRunTimeStats → std dev ✅ 1 kHz SysTick
SW‑02 MQTT round‑trip dashboard→MCU < 500 ms @ LAN Timestamp at publish/ISR ✅ Wi‑Fi RSSI ‑57 dBm
SW‑03 OTA firmware update Triggered via Node‑RED, success rate > 95 % 20 cycles, verify CRC ✅ Uses Atmel WINC1500 secure OTA
SW‑04 RAM usage fits 14 kB ≤ 13 kB after build arm-none-eabi-size ✅ Freed audio stack
SW‑05 Flash usage fits 256 kB ≤ 220 kB image Linker map ✅ Leaves 20 % margin
SW‑06 Email alert reliability 100 % on “attention” event 50 door‑open cycles ✅ SMTP relay via Node‑RED
INT‑01 System boots on battery only 3.4 V → 4.2 V Power‑cycle test ✅ Buck‑boost TPS61291

Legend 
✅ Met ⚠️ Partially met / needs improvement ❌ Not met

Validation Highlights

4. Project Photos & Screenshots

Final project, including any casework or interfacing elements that make up the full project

1

The standalone PCBA, top

2

The standalone PCBA, bottom

3

Thermal camera images while the board is running under load

4

The Altium Board design in 2D view

5

The Altium Board design in 3D view

6

Node-RED dashboard

7

8

Node-RED backend

9

10

Block diagram of your system

11

Codebase

Open-Source Components Used

This project leverages the following open-source components:

Acknowledgements