DALL·E 3 Integration: OpenAI Enhances ChatGPT with Visual Rendering from User Conversations
Oct. 20, 2023 -- OpenAI has announced the introduction of a new feature in ChatGPT that enables the creation of unique images based on user conversations. Available to Plus and Enterprise users, this feature facilitates visual rendering based on described visions and supports iterative refinements directly in the chat. The underlying technology is driven by OpenAI's advanced image model, DALL·E 3.
DALL·E 3 is the culmination of several research advancements both from within and outside of OpenAI. Notably, it outperforms its predecessor by producing visuals that are not just more appealing but also sharper. It exhibits a proficiency in rendering intricate components like text, hands, and facial features. Its enhanced capability to react to detailed prompts and support various aspect ratios stems from an advanced training regimen.
ChatGPT can now create unique images from a simple conversation. Image: OpenAI.
By employing a cutting-edge image captioner, better textual descriptions were generated for training images. Subsequent training of DALL·E 3 on these enhanced captions led to a model that aligns more closely with user-provided descriptions. A comprehensive exploration of this process is available in OpenAI's research paper.
Safety Measures
In deploying DALL·E 3, OpenAI has instituted a robust safety mechanism to curtail the production of harmful imagery, which includes content that is violent, explicit, or promotes hatred. Preemptive safety evaluations are conducted on user inputs and their corresponding outputs. Feedback from preliminary users and expert evaluations informed refinements, especially in identifying and addressing blind spots in the safety checks.
The model's propensity to generate images in the style of living artists, or those of public figures, has been curtailed, and demographic representation in the imagery has been enhanced. Comprehensive details on DALL·E 3’s deployment preparation can be found in the DALL·E 3 system card.
User Collaboration
OpenAI emphasizes the value of user feedback in refining its offerings. ChatGPT users can directly communicate with the research team to report concerns or discrepancies in output. This feedback loop, complemented by a broad user community, is instrumental in ensuring the responsible evolution of AI systems, aligning with OpenAI's mission.
Provenance Classifier
OpenAI is also piloting a provenance classifier designed to ascertain if an image has been produced by DALL·E 3. In early internal evaluations, it is over 99% accurate at identifying whether an image was generated by DALL·E when the image has not been modified. It remains over 95% accurate when the image has been subject to common types of modifications, such as cropping, resizing, JPEG compression, or when text or cutouts from real images are superimposed onto small portions of the generated image.
While the classifier indicates the likelihood of DALL·E 3’s involvement, it doesn't provide conclusive evidence. As part of broader efforts to discern AI-generated content, this tool, in conjunction with other strategies, could play a pivotal role in the future.
Artistic Integrity
Lastly, DALL·E 3 is programmed to reject image requests mimicking the style of living artists. Moreover, artists are provided with the choice to exempt their creations from being used in training subsequent image generation models by OpenAI.
Source: OpenAI
