Pawframe

Introduction

Unpeil introduces Pawframe, a groundbreaking technology that exports key frames based on pet facial expressions to extract high-quality labeled image data, providing an image dataset ideally suited for AI processing.

Background

In the current era of generative AI, where creative outputs are booming, the development of Large Multi-Modal models—capable of processing not just text but also images, videos, and sounds as inputs—has become increasingly active. Unlike traditional models like GPT-3 and Stable Diffusion, which rely solely on text inputs, these new models utilize diverse data formats.

Most multi-modal models that take videos as input currently adopt a sequential screenshot method, capturing images at predetermined intervals without considering the content of the video. In contrast, Unpeil's Pawframe technology codes the threshold points of pet facial expressions in the same environment, allowing it to select and extract qualitatively superior images as key frames.

Product

Engineering

Pawframe transforms pet videos into a structured image set through a three-stage process:

Segmentation: In the first stage, the system detects and segments objects and motions within the video. This step ensures precise identification of relevant elements, focusing specifically on the pet's movements and expressions.
FACS Labeling: After recognizing the pet's face, Pawframe applies a Facial Action Coding System (FACS) developed by Unpeil. This system, built on the analysis of over 100,000 data points, is tailored to each different pet species. It labels facial expression thresholds with corresponding text annotations, and these labels are saved along with the captured images.
Database Storage: The labeled image set is then stored in a database, similar to a video summarization. This process creates a concise and organized collection of keyframes.

The summarized keyframes can be used as-is, or they can be further utilized in secondary creative processes such as AI modeling, machine learning, and the creation of Metadog. This approach allows for the efficient generation of additional AI processing data, enabling rapid learning and storage with a lightweight dataset. By limiting the number of redundant data points, Pawframe also helps prevent AI overfitting, ensuring robust and reliable model training.

Design

Pawframe is offered as an API, optimized to allow AI to understand and learn from everyday pet videos captured on an iPhone. It enables quick and lightweight customization of objects, making it easy to integrate with other software.

Through Pawframe, consumers can experience a world where their pets are digitally recreated, most notably in an online memorial park. We envision a future where your pet’s digital presence is indistinguishable from reality. Unlike pre-set motion graphics that merely apply a pet’s skin, Pawframe’s labeling data and advanced AI learning enable your digital pet to interact with you, follow physical engines, and provide comfort without any sense of detachment. This becomes even more effective when combined with ambience and physical engines in XR devices.

Customers input videos without any preprocessing and receive high-quality, video-summarized image sets ideal for generative AI processing. These summarized videos not only convey the overall content but also create significant added value in AI-generated outputs such as images, videos, MetaDog.

Unpeil uses Pawframe to develop an online memorial service for XR-Device. Pet owners can use it as content while their pets are with them, and as a space for comfort after their pets have crossed the rainbow bridge.

Innovation

Unpeil addresses the social challenges faced by pets, a technologically underserved group through innovative solutions.

Superior Technology: Pawframe leverages a species-specific pet FACS, developed using over 100,000 data points, to automate preprocessing for AI training. This advanced system enables the extraction of high-quality, labeled datasets from video inputs alone, streamlining the process of creating AI-ready data with minimal human intervention.
Cultural Symbiosis with Pets: As the demand for pet welfare grows, Pawframe addresses the need for fine-tuning technology specifically for pets. In many countries, pets do not have portrait or intellectual property rights, making it easier to collect and utilize data for AI training. Moreover, AI models refined for pets can be adapted back to human applications, fostering a deeper connection between human and pet technology.
Healing from Loss: Pawframe offers a groundbreaking approach to Pet Loss Therapy, particularly in high-GDP countries where pets are considered family members. By digitally recreating pets, Pawframe helps owners cope with grief, providing comfort and therapeutic effects through interactive, personalized experiences. The technology not only evokes nostalgia but also simulates "what if" scenarios, offering profound emotional support to those mourning the loss of a beloved pet.

Conclusion

Pawframe is an innovative AI technology designed to extract high-quality key frames from pet videos. Leveraging Facial Action Coding System(FACS) specifically trained on over 100,000 data points, Pawframe accurately identifies critical expression thresholds in pets, extracting the most impactful frames. Unlike traditional models that rely on sequential screenshots, Pawframe focuses on the qualitative extraction of images, optimizing them for AI processing and storage.

This technology empowers users—both pet owners and service providers—to obtain a refined, video-summarized image set without extra preprocessing, enhancing AI-generated content creation across images, videos, and other multi-modal applications.