Apple has stepped into AI image editing by releasing a groundbreaking open-source multimodal AI model. Developed in collaboration with researchers from the University of California, Santa Barbara, the MLLM-Guided Image Editing (MGIE) model promises to revolutionize how images are edited.
Simplifying image editing with natural language commands
The MGIE model enables users to edit images using simple text commands, akin to giving instructions to a Photoshop expert. Unlike traditional methods, which often struggle to comprehend brief instructions, MGIE leverages multimodal large language models (MLLMs) to simultaneously understand natural language prompts and image data. This innovative approach eliminates the need for detailed descriptions, resulting in more accurate and intuitive editing.
According to the research paper accompanying the release, existing AI image editing tools frequently fall short when interpreting human instructions. However, MGIE’s ability to grasp the context of expressive instructions sets it apart. For instance, when presented with an image of a pepperoni pizza and instructed to “make this more healthy,” MGIE intelligently infers that “this” refers to the pizza and interprets “more healthy” as adding vegetables. The result? A pepperoni pizza adorned with nutritious green toppings, transforming it into a healthier option.
Superior performance demonstrated
In comparative tests, MGIE outperformed other models by accurately implementing complex editing commands. For example, when instructed to “add lightning and make the water reflect the lightning” to an image of a forested shoreline, MGIE seamlessly incorporated the lightning and its reflection on the water’s surface. In contrast, competing models failed to capture the intricate details of the command, highlighting MGIE’s exceptional capabilities.
Apple’s foray into AI image editing marks a significant step forward in the field, demonstrating the company’s commitment to advancing AI technologies. While the release of MGIE is a notable development, industry observers speculate that this may be just the beginning for Apple in the realm of AI.
