Aksiio Areas of R&D Expertise: Relative localization

From additive manufacturing and aerial imagery to augmented reality and custom-fit production, the pace of technological innovation is quickening. Companies can no longer afford to skip a generation of technology and expect to remain competitive or capture market share.

Some businesses are adept at using technologies available to them to create and improve products. But many others are having difficulties because they rely on technology that cannot keep up with the fast-changing environment. The difference often lies in how companies approach research & development and software development which we discussed in our previous article.

Areas of R&D expertise

As a software consulting company, Aksiio specializes in AI/ML, Computer Vision, and 3D technologies. With 12+ years of practical experience in R&D and software development, we know first-hand how to solve complex challenges in niche areas. This article highlights the most recent niche R&D project Aksiio undertook and what our team achieved as a result.

Relative localization (by panoramas/photos)

Scene understanding is the process of perceiving, analyzing, and explaining 2D or 3D dynamic scenes observed through a sensor network. This process mainly includes matching the signal information from the sensor that observes the scene with the model that humans use to understand the scene.

Aksiio researched and developed a solution to create a walkable virtual tour from a set of 360° panorama images taken in different locations, such as apartments, houses, and workplaces. Scene understanding requires deep expertise in at least three complex fields: computer vision, cognition, and software engineering.

The first task was to search for connections between images in all directions, and the direction of movement as panoramic images have an omnidirectional field-of-view, varying in time and physical location. After building and working through multiple hypotheses, the team approached the challenge with a solution similar to visual Simultaneous Localization and Mapping (SLAM) that matches feature points of images.

The next step was to find the best way to extract feature points to find connections between images by testing different extractors on the dataset. After testing SIFT feature detection algorithm was chosen as we need quality key points and don’t need to use our pipeline in real time, so we can sacrifice speed for accuracy. To make sure that we can rely on the extracted points, our team used Lowe’s ratio test after the key points matching process to remove false matches. Unprocessed 360° panoramic images are unsuitable for the matching process as they are heavily distorted, so the engineers adjusted the panoramas and separated them into several viewpoints.

Visualizations made using the microservice.

Following optimization and filtering out false positives, the team determined the physical locations in each panorama to create a fully functioning 3D walking tour. Given that the physical restrictions imposed by the global pandemic led to a boom in virtual tours, this R&D enables new business models and online products to help companies stand out.

Future R&D in this area can involve using Internal Measurement Units (IMU) inside smartphones to determine the panorama position in the real world to enable users to capture panoramas directly and use machine learning to improve the accuracy of matches even further.