Adobe Cloak AI automatically removes objects from video

Special effects that let Hollywood filmmakers remove unwanted objects and people from their final shots could soon be available at the push of a button.

Adobe has unveiled a peak of its Cloak project, which enables automatic editing of footage that would take professionals hours to complete.

Users of the tool, which is currently under development, can select any item for deletion and smart software fills in the gap.

The company has not yet given any word on if or when Cloak will be built into its software.

Special effects that let Hollywood filmmakers remove unwanted objects and people from their final shots could soon be available at the push of a button. Adobe has unveiled a sneak peak of its Cloak project, which enables automatic editing of footage 

DEMO FOOTAGE 

During the demo footage, the system can be seen editing a series of short clips frame by frame.

In the first clip, Cloak erases a couple sharing a walk through a canyon, before taking away a bag strap from across one of the walker’s chests.

This is particularly striking, as the software never gets the chance to see what is behind it, effectively forcing it to make an imaginative leap. 

Cloak is later seen deleting moving vehicles from a busy road and removing all trace of a pole in front of a cathedral.

The technology is in the experimental stages, with no set plans to implement it.

But previous research projects debuted at last year’s Adobe Max did later become part of the firm’s software suite.

The San Jose firm gave the first glimpse of Cloak at the Adobe MAX Creativity Conference, held from October 16 to 20 in Las Vegas.

It works in a similar way to an existing feature for Photoshop called content-aware fill.

This lets you choose an element of a still image that you want to delete.

The software uses machine learning to analyse the surrounding image and fill in the background.

Cloak works on the same principle, but with the much more complicated task of moving footage.  

Speaking to Endgadget, Adobe research engineer, Geoffrey Oxholm, said: ‘It’s a fascinating problem.

‘Everything is moving, so even if you nail one frame, you have to be consistent. 

‘The saving grace is that we can see behind the thing we want to remove.

‘If you’ve got a microphone to remove, you can see behind the microphone.

Users can select any item for deletion and smart software fills in the gap

In a side-by-side comparison, cloak is able to erase all trace of the couple sharing a walk through a canyon

Users of the tool, which is currently under development, can select any item for deletion and smart software fills in the gap. During the demo footage, the system can be seen editing a series of short clips frame by frame, starting with this couple

‘We can do really dense tracking, using parts of the scene as they become visible. That gives you something you can use to fill in.’ 

The technology could benefit video producers on any budget, from big studios to individuals. 

One example could be removing drunk people or other unwanted intruders who ruin video clips of special moments, like weddings or birthdays.

The technology could benefit video producers on any budget, from big studios to individual

Cloak works on the same principle, but with the much more complicated task of moving footage. Here we can see the vehicles have been removed

Cloak works in a similar way to an existing feature for Photoshop called content-aware fill. This lets you choose an element of a still image that you want to delete. Here a trio of moving cars is selected 

During the demo footage, the system can be seen editing a series of short clips frame by frame.

In the first clip, Cloak erases a couple sharing a walk through a canyon, before taking away a bag strap from across one of the walker’s chests.

This is particularly striking, as the software never gets the chance to see what is behind it, effectively forcing it to make an imaginative leap. 

During the demo footage, the system can be seen editing a series of short clips frame by frame

Another example for home users could be removing drunk people or other unwanted intruders who ruin video clips of special moments, like weddings or birthdays. Here we can see the pole has been removed

The technology could benefit video producers on any budget, from big studios to individuals. Here a pole is ruining the shot of an impressive looking Cathedral

Cloak is also able to take away a bag strap from across one of the walker's chests from the first video clip

Cloak is also able to take away a bag strap from across one of the walker’s chests from the first video clip

This is particularly striking, as the software never gets the chance to see what is behind it, effectively forcing it to make an imaginative leap

This is particularly striking, as the software never gets the chance to see what is behind it, effectively forcing it to make an imaginative leap

It is later seen deleting moving vehicles from a busy road and removing all trace of a pole in front of a cathedral.

The technology is in the experimental stages, with no set plans to implement it.

But previous research projects debuted at last year’s Adobe Max did later become part of the firm’s software suite.

MIT AND GOOGLE REVEAL AI THAT CAN RETOUCH YOUR PHOTOS IN REAL TIME

A new system from MIT and Google can automatically retouch photos in the style of professional photographers. 

The program is efficient enough to run on phones and is so fast that it can display retouched images in real-time, making it possible for users to see the final version of the image while still framing the shot. 

Researchers from Google and MIT’s Computer Science and Artificial Intelligence Laboratory unveiled it this week at Siggraph, the premier digital graphics conference.

Spot the difference: The program is efficient enough to run on phones and is so fast that it can display retouched images in real-time, making it possible for users to see the final version of the image while still framing the shot

Spot the difference: The program is efficient enough to run on phones and is so fast that it can display retouched images in real-time, making it possible for users to see the final version of the image while still framing the shot

The system uses machine learning and was trained with 5,000 raw and retouched images. 

The key is that the bulk of the image processing is performed on a low-res image.

But rather than trying to get a high-res image from a low-res input, the system creates a formula for modifying pixels.

It groups pixels using a 16 by 16 by 8 grid, in which each face of the grid stands in for thousands of pixels in a high-res image.

Each cell of the grid contains information that determines modifications of the colour values of the source images.

Read more at DailyMail.co.uk