Virtual Furniture Arrangement

1AF X99 COVER ARPatently Apple posted a granted patent report back in April covering an Augmented Reality invention that they inherited when acquiring Germany's Metaio. The theme of the invention related to moving virtual furniture in the home or office before committing to its placement in real life. Today Apple was granted a second related patent from Metaio relating to two Augmented Reality applications. The first is for an "Interior Design Application" while the second is for a new "Movie Poster Application." Some of this invention may have ended up in Apple's new ARKit as the patent notes that it provides "tools for authoring."

If you saw Apple's first AR demo at this year's WWDC event then you know that when Apple's SVP of Software Craig Federighi moved virtual furniture around on a demo table, the crowd reacted with amazement at the accuracy of the virtual elements moving in realtime. This is one of the inventions that made that happen.

2ANX FINAL AR PATENTYesterday Patently Apple posted a report titled "IKEA is excited about their new Augmented Reality App coming to iOS 11 with its Stunning Furniture Placement Accuracy." Of course being stunned by the accuracy due to ARKit means that the accuracy came from AR patents like the one presented today that describes an interior design application.

Granted Patent: Method of Image Processing for an Augmented Reality Application

Apple's newly granted patent covers their invention relating to a method of image processing in an augmented reality application, comprising the steps of providing at least one image of a real environment and performing image processing in an augmented reality application with the at least one image employing visualization of overlaying digital information with visual impressions or the image of the real environment and employing vision-based processing or tracking. The invention also relates to a computer program product comprising software code sections for performing the method.

Accordingly, the present invention is capable to improve the performance and usability of augmented reality applications, such as authoring tools.3AF X99 AR GRANTED PATENT Particularly, the invention attends to the problem of user unsatisfied visualization of overlaying the virtual information or digital content to an image or view of a real environment in augmented reality or visual search applications that employ vision-based detection, localization or tracking solutions by adjusting parameters including their values and/or operating flow (workflow) of the vision-based processing or tracking according to the usage of the image processing or the visualization, according to a visually perceivable property of the digital information and/or the real environment, according to display devices, and/or according to the way of user viewing the visualization.

In Apple's patent FIG. 1 noted below we're able to see a flowchart of an exemplary augmented reality visualization approach according to the state of art. In step 1 an image of real environment is captured by a capturing device, particularly a camera. In step 2.a, vision-based processing and/or tracking is performed on the captured image. The virtual information provided in step 2.b is overlaid onto the image of the real environment in step 3. In another application, the virtual information provided in step 2.b may be overlaid onto the view of the real environment which is provided by an optical-see-through display device, as commonly known in the art.

The visualization of overlaying digital (e.g., virtual) information onto the image of a real environment could be used for different purpose. Augmented reality visualization is often used to enhance user's visual perception by superimposing virtual information or digital content to the real environment in a correct perspective.

For example, augmented interior design could superimpose a virtual sofa to the camera image of the real environment of the room in a correct perspective. This application could visually support furniture arrangement in the room without having to move them physically. For this, an accurate pose of the camera with respect to the room is mandatory.

Apple's patent FIG. 4 noted above illustrates a flowchart of a method according to an embodiment of the invention overlaying a virtual object (here a furniture, like a sofa) onto the image of the real environment in an augmented interior design application based on an embodiment of the present invention demonstrated in FIG. 2. The property of the virtual information, such as symmetry, behavior, color, etc., and/or its usage is checked in step 2.c, wherein the method in response to this information configures or adjusts the processing or tracking method accordingly. For example, in step 2.d the threshold value in the similarity measures of sum-of-squared differences is increased. In step 3, the virtual sofa is then overlaid onto the image of the real environment.

Related posts: