In this post, we will create a sophisticated home security system using Raspberry Pi and AI capabilities. Our system will identify family members, detect strangers, and identify package deliveries while sending real-time web notifications.

Hardware requirements

1. Raspberry Pi 5 ( this is the one I use by default)
2. Hailo 8L Raspberry Pi AI Kit ( available here )
3. NexiGo Webcam (great value for money with great features, check it out here )
4. An internet connection (luckily, the Raspberry Pi 5 comes with integrated Wi-Fi connectivity and has an Ethernet port)

Software Setup

First, let's set up our  Raspberry Pi with the necessary software . I will skip the OS installation because  the CanaKit Raspberry Pi 5 kit comes with an SD card with the OS pre-installed:

sudo apt update && sudo apt full-upgrade sudo apt install hailo-all git  clone  https://github.com/hailo-ai/hailo-rpi5-examples.git cd  hailo-rpi5-examples source  setup_env.sh ./compile_postprocess.sh pip3 install opencv-python-headless numpy supervision pushbullet.py face_recognition

Code

Now, let's create smart_security_system.py document:

import  cv2 import  numpy  as  np import  supervision  as  sv import  face_recognition import  time from  hailo_rpi_common  import  GStreamerApp, app_callback_class from  pushbullet  import  Pushbullet from  hailo_models  import  YoloV5PostProcessing

# Initialize Pushbullet for notifications pb = Pushbullet( "YOUR_API_KEY" ) # Load known faces known_face_encodings = [] known_face_names = [] def load_known_faces ( directory ): for  filename  in  os.listdir(directory): if  filename.endswith( ".jpg" )  or  filename.endswith( ".png" ):             image = face_recognition.load_image_file(os.path.join(directory, filename))             encoding = face_recognition.face_encodings(image)[ 0 ]             known_face_encodings.append(encoding)             known_face_names.append(os.path.splitext(filename)[ 0 ]) load_known_faces( "known_faces" ) # Initialize YOLOv5 object detection yolo_postprocess = YoloV5PostProcessing() @app_callback_class class SmartSecurityCallback : def __init__ ( self ):         self.last_notification_time =  0         self.face_locations = []         self.face_names = []         self.process_this_frame =  True def app_callback ( self, buffer, caps ):         frame = self.get_numpy_from_buffer(buffer, caps) # Resize frame for faster face recognition processing         small_frame = cv2.resize(frame, ( 0 ,  0 ), fx= 0.25 , fy= 0.25 )         rgb_small_frame = small_frame[:, :, ::- 1 ] if  self.process_this_frame: # Find all faces in the current frame             self.face_locations = face_recognition.face_locations(rgb_small_frame)             face_encodings = face_recognition.face_encodings(rgb_small_frame, self.face_locations)             self.face_names = [] for  face_encoding  in  face_encodings:                 matches = face_recognition.compare_faces(known_face_encodings, face_encoding)                 name =  "Unknown"                 face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)                 best_match_index = np.argmin(face_distances) if  matches[best_match_index]:                     name = known_face_names[best_match_index]                 self.face_names.append(name)






    
        











    




    
    

        
        


        
        
            



            




                


        
        self.process_this_frame =  not  self.process_this_frame # Detect objects (people and packages)         detections = yolo_postprocess.postprocess(frame) for  detection  in  detections: if  detection.class_id ==  0 :   # Person if "Unknown" in  self.face_names:                     self.send_notification( "Stranger detected" ) else :                     self.send_notification( f"Family member detected:  { ', ' .join( set (self.face_names))} " ) elif  detection.class_id ==  39 :   # Package (assuming class ID 39 for package in COCO dataset)                 self.send_notification( "Package delivery detected" ) def send_notification ( self, message ):         current_time = time.time() if  current_time - self.last_notification_time >  60 :   # Limit to one notification per minute             push = pb.push_note( "Smart Security Alert" , message)             self.last_notification_time = current_time def get_numpy_from_buffer ( self, buffer, caps ): # Convert GStreamer buffer to numpy array # Implementation depends on the specific GStreamer setup pass def main ():     app = GStreamerApp( "Smart Security System" , SmartSecurityCallback())     app.run() if  __name__ ==  "__main__" :     main()
        

        
        
            
                

                

            
            

    
    

        


    
    
        
        
        

Many people asked me how to convert GStreamer's cap buffer to a NumPy array, so here I share my solution, especially if the cap is a video:

import  numpy  as  np import  gi gi.require_version( 'Gst' ,  '1.0' ) from  gi.repository  import  Gst def get_numpy_from_buffer ( self, buffer, caps ): """     Convert GStreamer buffer to numpy array     :param buffer: Gst.Buffer     :param caps: Gst.Caps     :return: numpy.ndarray     """ # Get the Gst.Structure from the caps     structure = caps.get_structure( 0 ) # Get the width and height of the video frame     width = structure.get_value( "width" )     height = structure.get_value( "height" ) # Get the pixel format (assuming it's in caps)     format_info = structure.get_value( "format" ) # Map the buffer to memory     success, map_info = buffer. map (Gst.MapFlags.READ) if not  success: raise  ValueError( "Could not map buffer" ) try : # Get the data from the mapped buffer         data = map_info.data # Determine the data type and shape based on the pixel format if  format_info ==  "RGB" :             dtype = np.uint8             shape = (height, width,  3 ) elif  format_info ==  "RGBA" :             dtype = np.uint8             shape = (height, width,  4 ) elif  format_info ==  "GRAY8" :             dtype = np.uint8             shape = (height, width) elif  format_info ==  "GRAY16_LE" :             dtype = np.uint16             shape = (height, width) else : raise  ValueError( f"Unsupported format:  {format_info} " ) # Create numpy array from the buffer data         array = np.ndarray(shape=shape, dtype=dtype, buffer=data) # Make a copy of the array to ensure it's not tied to the original buffer return  np.array(array) finally : # Unmap the buffer         buffer.unmap(map_info)





    

    




    

    
    


    
    

    
    

    
        
    
    
        

        
        
        


        


        


        


        
            
        
        

        
        
        
    
    
        

This implementation does the following:

1. It extracts the width, height and pixel format from the GStreamer Cap.
2. It maps the buffer into memory to access its data.
3. Depending on the pixel format it determines the appropriate numpy data type and shape.
4. It creates a numpy array from the buffer data.
5. Finally, it returns a copy of the array to ensure it is not bound to the original buffer.

Note that this implementation assumes certain pixel formats (RGB, RGBA, GRAY8 GRAY16_LE). You may need to add more format handling, depending on your specific use case. Also, make sure you have the necessary GStreamer and numpy dependencies installed:

pip install numpy PyGObject

You may also need to install the GStreamer development libraries on your system. On Ubuntu or Debian, you can do this with:

sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev

How it works

1. Face Recognition: System Use face_recognition library to recognize known faces. You need to populate the known_faces Table of contents.
2. Object Detection: We use Hailo 8L accelerated YOLOv5 to detect people and packages in video streams.
3. Notifications: When the system detects a stranger, family member, or package delivery, the system sends a web notification via Pushbullet.
4. Performance: By leveraging the Hailo 8L AI accelerator, we achieve real-time processing of video feeds, ensuring fast response times.

Setting up the system

1. Will "YOUR_API_KEY" Replace with your actual Pushbullet API key.
2. Add photos of family members toknown_facesDirectory, and use the person's name (e.g.john.jpg) Name each file.
3. If necessary, depending on the specific model you are using on your Hailo 8L, adjust YoloV5PostProcessing parameter.

in conclusion

This smart home security system demonstrates the power of combining a Raspberry Pi with AI capabilities. The Hailo 8L Raspberry Pi AI Kit provides the processing power needed to run complex AI models in real time, while  the NexiGo webcam ensures high-quality video input.

By building this system, you’ll not only enhance your home security, but you’ll also gain valuable experience with AI and computer vision. The possibilities for expansion are endless—you can add features like intruder alarms, pet detection, or integration with smart home devices.

Remember, the key to a great DIY project is choosing the right components. The Hailo 8L  kit and  NexiGo camera offer an excellent balance of performance and value, making them ideal for this project.