Home / Software & Service News / Facebook’s new mobile AI can process video in real time

Facebook’s new mobile AI can process video in real time

Facebook has started rolling out its “Caffe2Go” AI platform that does advanced style transfer video effects in real time using only your iOS or Android smartphone’s horsepower. While the painterly effects are cool (see the video, below), the tech behind it is much more interesting. Deep learning normally requires content “be sent off to data centers for processing on big-compute servers,” Facebook wrote, but with Caffe2Go, the processing can be done “in the palm of your hand.”

The new platform is part of a larger AI effort that includes the machine-vision Lumos app used to suss out images that violate its community standards. It has also open-sourced similar tech on Github to non-Facebook developers. It’s not the only company doing AI projects, of course. Google released its Tensorflow framework to the open source community and Microsoft recently made its Cognitive Toolkit available to developers.

Facebook first flaunted Caffe2Go last month, then brought some of the effects to a new camera in a limited European release. Much like the Prisma app, it transfers styles from Van Gogh or Monet onto any still or moving image. Processing live video normally requires at least a well-equipped PC, but Facebook says “we were able to provide AI [processing] on some mobile phones at less than 1/20th of a second,” six times faster than an eye blink.

Company engineers had to design software that worked with your smartphone’s limitations on memory and computing power. At the same time, they wanted the app to scale up for use on servers or workstation class machines. To that end, the team created a lightweight UNIX-based system 100 times smaller than similar deep-learning programs that works on CPU, GPU, Android and iOS. They then created add-in modules, including a CPU feature called NEON that improves mobile processing speeds.

Facebook is giving developers access to Caffe2Go via its stack, and plans to open-source parts of it “over the coming months.” While the video style transfer is a good test of the technology, it’s capable of doing other AI processing tasks related to image, speech and more. It won’t result in WestWorld any time soon, but it should open up the possibilities of what you can do on your smartphone in the near future.

Source: Facebook

Click Here For Original Source Of The Article

About Ms. A. C. Kennedy

Ms. A. C. Kennedy
My name is Ms A C Kennedy and I am a Health practitioner and Consultant by day and a serial blogger by night. I luv family, life and learning new things. I especially luv learning how to improve my business. I also luv helping and sharing my information with others. Don't forget to ask me anything!

Check Also

What we’re watching: ‘Raw’ and ‘Feast of Fiction’

Welcome back to Video IRL, where several of our editors talk about what they've been watching in their spare time. This month we're kicking things off with some seasonally-appropriate horror fare, that you can catch right away on Netflix or Amazon Prime. Then it's time for a Gundam throwback before Kris Naudus points out a couple of YouTube food channels perfect for binge eating or binge watching.

Them / Raw


Timothy J. Seppala

Timothy J. Seppala
Associate Editor

To get into the Halloween spirit, I've been watching at least one horror movie a day since the end of September -- the lower the budget, the better. Problem is, so many of the American low-budget or indie horror offerings on Amazon and Netflix are crappy Paranormal Activity clones, cheap-thrill gore-fests or uninspired found-footage "documentaries." Whether it's by design or coincidence, I've found that French horror movies have held my attention the most lately. Specifically, 2016's Raw, as well as Them, from ten years prior. They're more psychological thrillers than straight-up horror, but that didn't stop me from being more on edge while watching them one afternoon than I was during A Haunting in Saginaw, Michigan, late at night. Both start with a car crash, but they couldn't finish any more differently.

Raw, recently added to Netflix, tells the tale of a vegetarian girl in her first week at a prestigious veterinary school. During a hazing ritual, she's forced to eat a raw rabbit kidney. She immediately gets sick, throws up and wakes herself up that night scratching a full-body rash to near bleeding. This bout with food poisoning is just the beginning, though, and soon protagonist Justine finds out she has a taste for forbidden fruit. As the remaining 70-ish minutes unfolded, I lost track of how many times I clasped my hands over my mouth, agape in shock, to stifle my shouts of "OHMYGODWHATTHEFUCKISEVENHAPPENING?!"

But French director Julia Ducournau balances every body-horror scene either with something pedestrian twisted into being unsettling (like a horse on a treadmill) or with something that makes you ask how far Justine can go before someone confronts her about her new diet. And those questions keep coming right until the credits roll. I can't say I enjoyed watching Raw, but it was a hell of a ride.

The same goes for Them, currently streaming on Amazon Prime. Its focus is narrow, centering on a young couple living in a cavernous farmhouse, terrorized over the course of a night by unseen horrors. The camera never quite gives away who (or what) the perpetrators are, and revealing the twist would be a sin. As with Raw, its atmosphere and overall creepiness won me over straightaway. The scariest part? Realizing that I've probably driven past a shot like the final scene countless times and not thought twice about it. If you're willing to read subtitles, both of these should make you shiver and scream more than The Conjuring 2 on HBO Go could ever hope to.

Mobile Suit Gundam The 08th MS Team


David Lumb

David Lumb
Contributing Editor

I'd heard that a lot of anime had left Hulu, but I scanned their selection anyway looking for classic shows I'd missed, like the original Mobile Suit Gundam. They don't have that -- but they did have a series I didn't finish the first time it aired on Toonami, the 1996 classic Gundam side story The 08th MS Team. Unlike the franchise's other show released the year before, the massively successful Gundam Wing, 08th ditches the brand's typical pretty-boys-in-unbeatable-robots for a grounded and sobering story about the people who get caught up in wars -- desperate soldiers, civilians and guerrillas alike. It's dirty, honest, utterly humane and gorgeously animated.

It's also a little preachy and melodramatic, and it shows its age with odd sexist moments. While it's still the Thin Red Line of the Gundam universe, I remember it far more fondly from when my 14-year-old self grazed the series on its first American airing. There's something sad in seeing an old favorite for the flawed media it always was. Much like Waypoint's Rob Zacny, I've grown up and seen a lot since I first caught the show as a starry-eyed teen. I still think The 08th MS Team is a wonderful little 12-episode miniseries with a big heart, but I won't revere it so highly -- and will think a little harder about who I recommend it to.

Feast of Fiction / Binging with Babish


Kris Naudus

Kris Naudus
Senior Editor, Database

Back in March, I came home from a trip only to discover that my oven didn't work. The cooking gas in my building had been shut off due to a leak. My building management seemed to be on it, so I made do with a combination of microwavables, toaster oven and Seamless. Unfortunately, weeks and months went by, calls to the city were made and permits were issued, but, even as I write this in October, gas still has not been restored to my building. My landlords eventually threw their collective hands in the air and began installing electric ranges in every apartment, so a few weeks ago I was finally able to cook for myself again.

I am so jazzed to be able to make food. Hot food! Scrambled eggs! Steak! Cookies! I started reading food blogs and cookbooks, and shopping to refill my pantry. I'm halfway through Kenji Alt-Lopez's The Food Lab, a huge 900-page hardcover that talks about the science of how food cooks. On the lighter side, I've also been reading food-themed comics like Delicious in Dungeon and Food Wars. And the latter title (which is also an anime) ended up sucking me into a YouTube hole of food videos that I've been obsessed with ever since.

You see, the very first chapter of Food Wars features the "Gotcha" Pork Roast, a bacon-wrapped potato loaf that hero Soma Yukihira makes to save his family restaurant. It looks pretty tasty, so I searched for recipes and pics online and stumbled onto Jimmy Wong and Ashley Adams' Feast of Fiction, a series that demonstrates how to make various foods seen in cartoons, video games and comics. If you ever wanted to taste Steven Universe's beloved Cookie Cat ice cream sandwiches or Kirby's super-spicy curry, there's an episode for you. One thing I really enjoy is how they also incorporate crafts into it, showing how to make paper wrappers for your Reptar chocolate bars or genuine-looking Ecto Cooler Hi-C boxes.

I've been marathoning through the episodes, which the YouTube algorithms have definitely picked up on at this point, throwing food show after food show into my suggestions. One that caught my eye was Binging with Babish. Where Feast of Fiction mostly sticks to the realm of kids' cartoons, anime and video games, Binging with Babish is a little more mainstream, covering foods from popular media like Mad Men, Seinfeld and House of Cards. Still, there's a bit of overlap -- both Babish and Feast have done their own takes on the Ultimeatum from Regular Show and Krabby Patties from SpongeBob SquarePants. But the recipes are different, and I watch the shows for the personalities. Feast of Fiction is pretty silly (and there's a cute dog), while Binging with Babish is a little more subdued. Not that Babish can't be ridiculous as well -- the Moist Maker is one of the most ridiculously complicated sandwiches I have ever seen, basically asking you to cook an entire Thanksgiving dinner.

Sadly, I still haven't done a lot of actual cooking since getting my stove back. I'm having too much fun watching other people do it instead, with the added bonus that I don't have to clean up the mess.

"IRL" is a recurring column in which the Engadget staff run down what they're buying, using, playing and streaming.

css.php