87% of this Image is AR & AI, 13% is Human
This photoshoot features real people wearing Augmented Reality clothing with Generative AI editing. Read on to learn how the photoshoot was done, the takeaways, and the fashion use cases of AR and AI.
Highlighted in red is the 13% that is human and untouched by AI or AR. It’s shocking how realistic some of it is. Read on for the details!
Starters (Some Definitions)🍸
Augmented Reality = filters. It has evolved from early applications like Pokemon Go to more interactive experiences, including face filters (Snapchat) and clothing filters, which read and overlay garments onto the user, providing a simulated 360-degree view akin to wearing the garment in the physical world.
Digital Fashion involves creating garments as 3D models, ranging from simple to complex designs that push creative boundaries and are intentionally impossible to produce in the physical world; these "digitally native" pieces exist in 3D software, enabling designers to break free from traditional constraints, and are praised for democratizing fashion design by bringing a library of materials and textures among other resources to the fingertips of anyone with access to a computer and internet.
Generative Artificial Intelligence, represented by ChatGPT and its creative companions Midjourney and Dall-e-2, functions as a low-level assistant. Imagine that every piece of public data that the web has accumulated over the past several decades is stored in a library. Instead of the librarian just being able to retrieve books for a research project upon request, she can take all the books, process all of that information, and synthesize it into a final report. When you use AI, you are collaborating with the world’s data.
Takeaways
AR can provide body type data improving AI styling and offering a smarter online and in-store shopping experience.
AR try on when combined with AI recommendations promotes body positivity.
Implementing a digital prototyping process with generated variations and AR try-on before sample production is more efficient and reduces waste.
AR technology allows enthusiasts to rent digital replicas of iconic red carpet and archival outfits, enabling them to recreate "celebrity-level" photos.
Since AI has little to no learning curve, non-technical creators can use it to efficiently and effectively communicate their vision to designers via generated visuals or create high quality assets on their own.
Digital fashion, AR try-on, and generative AI editing are shockingly realistic especially when strategically combined with real, non-digital elements. Only 13% of the “Summer of AI” image is human or non-digital.
Over time, we can expect the emergence of specialized AI tools, like OpenAI's recent unveiling of GPTs, designed for specific tasks. Future interaction with AI will consist of combining various tools, blending their strengths for optimal results.
Luxury fashion brands now have new ways of offering specialty experiences. AI can be used to generate one of a kind designs that resemble a collection. And physical garments can have an added AR experience.
Websites will learn our navigation behavior and remove unused features or options to streamline consumer attention.
UseCases
Opportunities for AI
Design Tool for Non-Graphic Designers: As seen in the photoshoot above, AI empowers amateur photographers and creatives to generate professional-quality images. This provides an opportunity for small and medium brands to create product photoshoots, artwork, social media assets, and more that rival those of bigger brands with larger budgets, offering an accessible and empowering design tool without a steep learning curve. This is particularly valuable to the small business owners who are stretched for time and have to manage the high quality content expectations of today’s consumers.
AI that Perfectly Reads Your Mind… Is Coming: We are waiting for the day that a few simple commands could spit out exactly or very close to what we envision. Despite the impressive images generated by Dalle-2 and Midjourney, new AI tools, such as Welcome AI and specialized tools like the font generator in Adobe Express, are emerging to provide more accurate translation of a creator's sketches or written direction. The next generation of AI tooling will be specialized allowing a creator to mix and match the various strengths of each tool.
Test Ideas At the Speed of the Internet: AI serves as a valuable tool for visually testing and refining creative ideas while in the discovery and ideation stages. Tools like Midjourney provide fast concept visualization and introduce new ideas, accelerating the creative process. Ultimately, creatives can include visuals when communicating among collaborators, improving the communication and offering a foundational image for graphic designers to refine and edit from.
Photoshop can be as Easy to Use as Canva: Demonstrated in this photoshoot, editing with generative AI enables users with limited design tool experience to effortlessly correct specific elements, seamlessly generate missing portions, and add new elements (more details on that in the instructions section to follow). In this case, AI removed barriers to achieving the creative vision by compensating for our lack of professional photography skills. It has dramatically improved access to advanced capabilities in tools like Photoshop and has the potential to make using its advanced design features as simple as using Canva.
One of a Kind AI Gen Graphic Streetwear: AI presents a fresh avenue for luxury brands to provide exclusivity via unique AI-generated graphic designs for clothing. These final garments maintain a unified collection and brand identity while granting consumers a one-of-a-kind product. While applicable to all brands, this opportunity is particularly intriguing for streetwear brands where graphic tees and hoodies flourish.
Online Stores Personalized to your Navigation Behavior: AI's role in online shopping lies in personalized experiences where websites anticipate and customize navigation based on individual behaviors. This reduces the clicks needed to access a product, streamlines the process by limiting overwhelming options, and ultimately increases both shopper retention and completed purchases. This concept is more noticeably applied on sites that we visit regularly such as gmail where certain unused functions specific to the user can be removed for a less overwhelming and more streamlined experience. With a consumer’s attention focused they are more likely to do the intended action and convert.
Opportunities for AR
AR Try on & AI Variants to be Part of the Prototyping Process: Combining digital fashion with AR allows companies to virtually test garment fit, color matching, and appearance on real people before moving to sample production. These insights are achieved in less time than producing a sample and could cut down the amount of samples needed (reducing material waste) before moving the garment to production. The resulting AR try on images can be used for believable product pages to gauge consumer interest, strategize items in a collection, and fundraise via pre-orders. Experts are advancing AR tools and digital texture libraries to mimic physical garment wearing, promising a bright future for the technology’s upcoming applications. Startups, such as VopplAR, use AI for generating rapid material, color, and design variations before AR try-on. The entire fashion design process, from sketch to 3D modeling to AI iterations and then AR try-on, can often be completed within a day, as indicated by Co-Founder Nikki Salami. Not to mention, digitization of the design process is a step towards sustainability, helping large and small companies quantifiably work towards ESG goals.
Digital Fashion & Photography = New Art Form: AR and digital fashion serve as an artistic medium, merging real models with unworldly clothing designs to create a photorealistic art form. This intersection allows creatives to explore the blend of reality and digital fashion, telling compelling stories through cutting-edge photography techniques while showcasing advanced fashion designs or wearable art. With free apps like Zero10, DressX, and Geenee AR, photographers can easily experiment with the existing digital fashion pieces on the apps. For digital fashion designers, photographs offer a potential source of income beyond selling garments on digital marketplaces. Incorporating technology whether AR or generative AI in traditional art forms such as fashion design, photography, or even painting will become more common. The combination tells a story of the collaboration between people, technology, and art.
Try on Celebrity Red Carpet or Archival Looks: AR try-on enables anyone to capture an iconic look in a picture. A digital replica of a Red Carpet dress in AR adjusts to any body size without alterations, allowing all people to wear garments that are typically made for a specific body type. For lesser known influencers and models, AR try-on broadens their high-end outfit options for rent, allowing them to achieve a "celebrity-level" appearance. Selling access to try on these high-end pieces could be a standalone business model for luxury brands. For example, Bob Mackie could offer enthusiasts the opportunity to rent a digital replica of a 1957 Marilyn Monroe dress for $50, enabling them to create photoshoots that rival Kim Kardashian's iconic appearance at the Met Gala in the physical dress. Although the technology isn't quite there yet, this application could become common in the next few years.
Improved AI Styling & Shopping through Snapchat Filters: AR clothing try-on is currently on select product pages and is set to expand among online retailers. Already shown to improve sales, AR try-on displays how garments look on a customer’s body. As usage grows, AR will gather body type data, enhancing AI styling and offering personalized clothing suggestions for immediate try-on, mirroring in-store experiences. Leveraging diverse body type data, the AI model acting as a stylist can deliver improved estimated fit scores, precise size recommendations, and improved clothing selections, fostering a more personalized shopping experience that minimizes returns and prioritizes inclusivity. AR try on coupled with AI styling could revolutionize online shopping to be a version of Snapchat with clothing as filters and each new filter being an AI stylist recommendation that builds on the previous.
Smart Stores Coming Right Up: Smart retail experiences extend to physical stores through smart mirrors, eliminating the need for dressing rooms by allowing customers to try on clothing effortlessly. The mirror recognizes garments held in front of it and displays their digital versions on the customer's reflection. AI styling offers real-time suggestions, while the mirror serves as a one stop shop for checking item availability and facilitating checkout. This efficient system allows store associates to focus on fashion and providing store visitors with artistic recommendations to complement the AI model.
Limitless AR Features to Physical Garments: Garments can exist in their physical form with limitless AR features, such as animated angel wings, rhinestones, or flowing water. For instance, a simple white cocktail dress can have wings by scanning a QR code via a smartphone to activate the AR filter. This use case caters to hype brands with a strong influencer following, as the AR-enhanced experience is appealing for social media pictures and videos. AR experiences could revolutionize fashion shows by presenting a dual showcase of physical items and their AR enhanced versions. Future advancements like Apple Vision Pro aim to streamline these AR experiences, potentially allowing users to access these filters by hovering over the garment without the need for QR codes.
The Process Behind the Photoshoot 🎞️
Step 1, Selecting the AR Fashion App: I examined three apps, each with free prebuilt digital fashion options, focusing on those with the most variety and interesting choices.
Geenee AR may not offer as many options, but it's important to highlight that they are pioneers in realistic body mapping algorithms, prioritizing the lifelike imitation of wearing clothing in real life. Their innovation is something to be on the lookout for in the coming years. Their outfits can be accessed through their website, where these options are available for try-on via a QR code.
DressX offers the largest variety with several physical brand crossovers such as Tommy Hilfiger and Mugler, though most options are paid. Their outfits can be accessed through their app.
Zero10 🥇has similar physical brand crossovers, but it stood out for its extensive collection of free avant garde options. In the spirit of keeping this project cost efficient and exploring the complex nature of digital fashion, I went with this option. Their outfits can be accessed through their app.
Step 2, Pick a Location, Get Some Friends, & Snap Some Pics: Contrary to my expectations, location didn't matter as much as good lighting did. The background didn’t matter thanks to background removal editing. We ended up taking pictures in the Hudson Yards mall and some right outside. I highly recommend bringing along a friend who takes good pictures, shout out to Areeba, her photo taking abilities made a difference! After changing into some Nike Pros and sports bras (to minimize the amount of editing), we selected outfits in the Zero10 app and snapped some pictures. Make sure to bring a ton of positivity, taking pics in the middle of a mall takes courage!
In our exploration of Zero10's digital fashion collection, jackets consistently appeared the most realistic, adapting well to different arm positions with natural-looking folds akin to physical clothing. However, abstract digital pieces, while intriguing, appeared less convincing in their fit and were not chosen for the final image. Exploring paid digital fashion options with potentially superior fit quality stands as an intriguing avenue for further investigation.
Step 3, AI Generated Background and Text: For the background creation process, I considered both Dall-e-2 and Midjourney, ultimately choosing Dall-e-2 for its strength in producing photorealistic images, which aligned with the goal of achieving a realistic photoshoot. Midjourney, on the other hand, produced cartoonish, graphic, and imaginative outputs.
Entering this step during the early ideation phase I didn’t have any ideas for what the background should be. I played around with various prompts and let the options that Dalle-2 presented to guide and inspire me. After 45 minutes, I had a clear vision, and 15 minutes later, I had a ready to use graphic that fulfilled my requirements.
It took approximately 130 prompts to reach the winning prompt, which was "futuristic sidewalk." Interestingly, specificity proved counterproductive with Dalle-2, as it produced better results with a more open and ambiguous prompt, as demonstrated by the winning prompt. Additionally, a large number of prompts appeared to assist Dalle-2 in better learning my preferences, leading to a highly satisfactory output.
For the text element, I utilized the Adobe Express Font Generator, my favorite AI generator thus far - the outputs are almost always highly satisfactory. This tool generates text fonts based on prompts, allowing for creative options like "fuzzy” letters or, in my case, "carbon-colored deflated balloon” letters. After typing a prompt and exploring various options, the generator proved highly responsive to specificity, providing a large range of accurate font choices. Quickly visualizing the fonts with my preferred text, I generated additional variations for more granular perfection and efficiently developed a custom font within minutes. I saved the complete title and repeated this entire process with the prompt “White cloud of gas” for the subtext.
Step 4, Generative AI Editing & Piecing it All Together: Next up was editing the models in the AR clothes. The first step was removing the background from each picture which happened in Photoshop Beta with 3 simple clicks, select object, select inverse, delete. Some pictures required little to no AI correcting, but some were mistakenly taken waist up or had missing portions. #5 required generation from knee down. #6 had a shoe that was out of frame and required generation. #2 and #4 required some outfit generating. #1, #3, #7 required no edits.
These parts were created using Photoshop Beta. Initially, I selected the approximate size of the missing area using a rectangle selector below the image. Then, I used generative fill, entering prompts that provided three options, often matching seamlessly with the existing image. However, some options didn't match the outfit style, so I alternated between generating more options and adjusting the prompts until reaching a satisfactory outcome. For #5, I used the prompt "khaki pants with sneakers." Although the generated area comes with a background, it’s easily removable using select object, select inverse, and delete functions. Similarly, for #6, I generated sneakers using prompts like "front of white sneakers," "matching sneaker," "white Jordans," and "tip of sneaker." To tackle a generating part of something like the front of the shoe, I varied the selected area to include the other shoe and part of the pants, providing the AI with more contextual information.
***It's important to note that each generation results in a unique outcome; using the same prompts on the original image produces different results each time.
For #2 and #4, I experimented by elements like jewelry to assess the realism of adding Gen AI elements. In #2, the original AR dress was strapless, revealing the single shoulder sports bra worn during the photoshoot. I used generative fill to substitute the blue sports bra with a matching "shimmery metallic single shoulder top" and included a "silver chain necklace" to enhance the outfit's coherence. To edit around the feathery red section, I used the lasso tool to select the areas above and below the red portion before beginning to generate.
For #4, the AR outfit included pants and a top, with the pants fitting well but the top not matching properly. To rectify this, I used generative fill to substitute the ill-fitted top with a better matching "navy puffer jacket." However, generating the midriff area took several attempts to match the skin tone accurately. During the duration of this project, an Adobe Beta update implemented restrictions on skin-related prompts, hindering the effectiveness of generating skin-related elements. Although I appreciate Adobe's effort to regulate generative AI use, these limitations affected the creative process, leading to the selection of a top that covered the midriff for the final image due to the challenges with generating skin-related elements.
It's important to highlight that Photoshop Beta encounters difficulties in generating heels. Despite multiple attempts to generate heels for #2 and #3, Photoshop Beta couldn't successfully generate them. While I was impressed by Beta's capabilities in generating images from simple prompts, I had to perform some manual touch-ups using a pixel art brush. This indicates that AI won’t be able to do it all and there will always be a need for manual editing especially to achieve a specific desired outcome.
Once all of the model images were edited, I put the background, text files, and model images together into the final result you see here. That’s it!
The Juicy, Where Did This Project Come From 🧃
This project was initiated by various influences, starting with an engagement in AR while organizing a digital fashion pop-up during Cornell Fashion Week and Girls Who Code Code Day. The experience of showcasing a digital fashion designer's pieces in AR, sparked the interest to dig deeper. Another significant moment was encountering the Zero10 pop-up with Coach in Soho, prompting curiosity about the realism of AR with clothing beyond handbags and the potential evolution of AR try on.
Joining forces with Michelle Collins, President and Chief Experience Officer of A Non-Agency, to create an AI Fashion Technology Artistry program also greatly shaped the perspective seen throughout this article, aligning with her ideologies on AI and its empowerment of global communities. The rise in TikTok videos testing Photoshop beta and the popularity of AI, particularly Dall-e-2 and ChatGPT, further influenced the decision to combine AR and AI touchpoints, revealing a natural synergy and more exciting potential than originally anticipated.
Throughout the process of image generation, editing, and composition, the rising marketing influence of Barbie played a role, evoking a sense of nostalgia and inadvertently reflecting a collection of Barbie dolls in the resulting images. In a way this project is also a tribute to Aqua’s Barbie Girl song. My favorite line of that song since I was a little girl is, “Imagination, life is your creation” because it always made me feel like my imagination could take me anywhere, and here I am, imagining the future of fashion and creating it with emerging tech.
Acknowledgements go to Ethan Igel, Jenna Winocur, Olohirere Longe, Meghana Gavirneni, Ann Xu, and of course ChatGPT 3.5 who served as sources of inspiration, editors, and vital contributors during the development of this thought piece.
From the inception of writing this article in August to its publication in December, multiple points highlighted earlier have been substantiated by various sources, including LinkedIn News and Vogue. For further insights into this field, McKinsey's comprehensive analysis of Generative AI in Fashion offers valuable perspectives.
Thanks for sticking around to the end :) Don’t hesitate to reach out with any questions!
Wow! This is interesting! The outcome is definitely worth the effort. Well done Karina.
The possibility with AR and AI is immense and I feel more innovations on the application of these technologies across different sectors will be made in the coming years. The fashion industry in quite unique as it involves everyone.
Thank you for sharing this project with us. I can see your passion through every detail of your write up. Cheers!