Discussing Prompt Engineering and recent OpenAI developments with ex-OpenAI Creative Apps and Scientific Communicator Andrew Mayne

Timestamps:

00:00:00 - Teaser Reel Intro

00:01:01 - Intro / Andrew's background

00:02:49 - What was it like working at OpenAI when you first joined?

00:12:59 - Was Andrew basically one of the earliest Prompt Engineers?

00:14:04 - How Andrew Hacked his way into a tech job at OpenAI

00:17:08 - Parallels between Hollywood and Tech jobs

00:20:58 - Parallels between the world of Magic and working at OpenAI

00:25:00 - What was OpenAI like in the Early Days?

00:30:24 - Why it was hard promoting GPT-3 early on

00:31:00 - How would you describe the current 'instruction age' of prompt design?

00:35:22 - What was GPT-4 like freshly trained?

00:39:00 - Is there anything different about the raw base model without RLHF?

00:42:00 - Optimizations that go into Language models like GPT-4

00:43:30 - What was it like using DALL-E 3 very early on?

00:44:38 - Do you know who came up with the 'armchair in the shape of an avocado' prompt at OpenAI?

00:45:48 - Did you experience 'DALL-E Dreams' as a part of the DALL-E 2 beta?

00:47:16 - How else has prompt design changed?

00:49:27 - How has prompt design changed because of ChatGPT?

00:52:40 - How to get ChatGPT to mimick and emulate personalities better?

00:54:30 - Mimicking Personalities II (How to do Style with ChatGPT)

00:56:40 - Fine Tuning ChatGPT for Mimicking Elon Musk

00:59:44 - How do you get ChatGPT to come up with novel and brilliant ideas?

01:02:40 - How do you get ChatGPt to get away from conventional answers?

01:05:14 - Will we ever get single-shot, real true novelty from LLM's?

01:10:05 - Prompting for ChatGPT Voice Mode

01:12:20 - Possibilities and Prompting for GPT-4 Vision

01:15:45 - GPT-4 Vision Use Cases/Startup Ideas

01:21:37 - Does multimodality make language models better or are the benefits marginal?

01:24:00 - Intuitively, has multimodality improved the world model of LLM's like GPT-4?

01:25:33 - What would it take for ChatGPT to write half of your next book?

01:29:10 - Qualitatively, what would it take to convince you about a book written by AI?  What are the characteristics?

01:31:30 - Could an LLM mimick Andrew Mayne's writing style?

01:37:49 - Jailbreaking ChatGPT

01:41:12 - What's the next era of prompt engineering?

01:45:50 - How have custom instructions changed the game?

01:54:41 - How far do you think we are from asking a model how to make 10 million dollars and getting back a legit answer?

02:01:07 - Part II - Making Money with LLM's

02:11:32 - How do you make a chat bot more reliable and safe?

02:12:12 - How do you get ChatGPT to consistently remember criteria and work within constraints?

02:12:45 - What about DALL-E?  How do you get it to better create within constraints?

02:14:14 - What's your prompt practice like?

02:15:10 - Do you intentionally sit down and practice writing prompts?

02:16:45 - How do you build an intuition around prompt design for an LLM?

02:20:00 - How do you like to iterate on prompts? Do you have a process?

02:21:45 - How do you know when you've hit the ceiling with a prompt?

02:24:00 - How do you know a single line prompt is has room to improve?

02:26:40 - Do you actually need to know OpenAI's training data?  What are some ways to mitigate this?

02:30:40 - What are your thoughts on automated prompt writing/optimization?

02:33:20 - How do you get a job as a prompt engineer?  What makes a top tier prompt engineer different from an everyday user?

02:37:20 - How do you think about scaling laws a prompt engineer?

02:39:00 - Effortless Prompt Design

02:40:52 - What are some research areas that would get you a job at OpenAI?

02:43:30 - The Research Possibilities of Optimization & Inference

02:45:59 - If you had to guess future capabilities of GPT-5 what would they be?

02:50:16 - What are some capabilities that got trained out of GPT-4 for ChatGPT?

02:51:10 - Is there any specific capability you could imagine for GPT-5?  Why is it so hard to predict them?

02:56:06 - Why is it hard to predict future LLM capabilities? (Part II)

02:59:47 - What made you want to leave OpenAI and start your own consulting practice?

03:05:29 - Any remaining advice for creatives, entrepreneurs, prompt engineers?

03:09:25 - Closing

Subscribe to the Multimodal By Bakz T. Future Podcast!

Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV

Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820

Google Podcasts -  https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s

Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future

Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml

Connect with me:

YouTube - https://www.youtube.com/bakztfuture

Substack Newsletter - https://bakztfuture.substack.com​

Twitter - https://www.twitter.com/bakztfuture​

Instagram - https://www.instagram.com/bakztfuture​

Github - https://www.github.com/bakztfuture

Sam and Greg back are at OpenAI - here's my message to them

Subscribe to the Multimodal Podcast!

Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV

Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820

Google Podcasts -  https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s

Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future

Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml

Connect with me:

YouTube - https://www.youtube.com/bakztfuture

Substack Newsletter - https://bakztfuture.substack.com​

Twitter - https://www.twitter.com/bakztfuture​

Instagram - https://www.instagram.com/bakztfuture​

Github - https://www.github.com/bakztfuture

Exploring AI Safety in Canada, how politics works, learning more about the AIGS Canada non profit.

(Video version on YouTube channel coming soon)

AIGS Policy whitepaper:

https://aigs.ca/advocacy/ 

AIGS Canada:
https://aigs.ca/

Mario Gibney: 

https://twitter.com/Mario_Gibney

https://www.linkedin.com/in/mario-gibney-08bb7b45/ 

Wyatt Tessari L’allié:

https://twitter.com/WyattTessari 

https://www.linkedin.com/in/wyatttessari/ 

Subscribe to the Multimodal Podcast!

Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV

Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820

Google Podcasts -  https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s

Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future

Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml

Connect with me:

YouTube - https://www.youtube.com/bakztfuture

Substack Newsletter - https://bakztfuture.substack.com​

Twitter - https://www.twitter.com/bakztfuture​

Instagram - https://www.instagram.com/bakztfuture​

Github - https://www.github.com/bakztfuture

My first impressions getting access to the exclusive DALL-E 3 release as an Official ChatGPT plugin. So far, I'm pretty impressed.

Tweet thread unboxing and exploring DALL-E 3:

https://twitter.com/bakztfuture/status/1709303961625264572

Subscribe to the Multimodal Podcast!

Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV

Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820

Google Podcasts -  https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s

Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future

Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml

Connect with me:

YouTube - https://www.youtube.com/bakztfuture

Substack Newsletter - https://bakztfuture.substack.com​

Twitter - https://www.twitter.com/bakztfuture​

Instagram - https://www.instagram.com/bakztfuture​

Github - https://www.github.com/bakztfuture

Why I am no longer optimistic about DALL-E ...

Goodbye DALL-E

Subscribe to the Multimodal Podcast!

Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV

Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820

Google Podcasts -  https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s

Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future

Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml

Connect with me:

YouTube - https://www.youtube.com/bakztfuture

Substack Newsletter - https://bakztfuture.substack.com​

Twitter - https://www.twitter.com/bakztfuture​

Instagram - https://www.instagram.com/bakztfuture​

Github - https://www.github.com/bakztfuture

Multimodal Returns!  

Topics I've covered:

• Personal life updates
• 2023 - the era we're in
• The Highs and Lows of AI Media
• OpenAI Superalignment Announcements

Timestamps:
[coming soon]

Subscribe to the Multimodal Podcast!

Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV

Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820

Google Podcasts -  https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s

Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future

Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml

Connect with me:

YouTube - https://www.youtube.com/bakztfuture

Substack Newsletter - https://bakztfuture.substack.com​

Twitter - https://www.twitter.com/bakztfuture​

Instagram - https://www.instagram.com/bakztfuture​

Github - https://www.github.com/bakztfuture​

Reviewing major events from the 2022 year with returning guest David Shapiro!

Timestamps:
00:00 Intro
04:58 #7 - Nvidia NeMo Megatron
15:00 #6 - Petals
23:24 #5 - Bloom
28:25 #4 - Meta AI wins at Diplomacy
41:18 #3 - Crypto crash!  FTX implosion
56:54 #2- DALL-E 2, Midjourney, Stable Diffusion
1:18:29 #1- ChatGPT
1:29:41 Honourable Mentions
1:56:01 How to Start a Startup Presentation
2:26:37 Post Presentation Discussion
3:09:45 Outro

David Shapiro:
https://www.youtube.com/c/DavidShapiroAutomator
https://www.patreon.com/daveshap
https://www.davidkshapiro.com/
https://twitter.com/dshap_automator

Subscribe to the Multimodal Podcast!

Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV

Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820

Google Podcasts -  https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s

Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future

Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml

Connect with me:

YouTube - https://www.youtube.com/bakztfuture

Substack Newsletter - https://bakztfuture.substack.com​

Twitter - https://www.twitter.com/bakztfuture​

Instagram - https://www.instagram.com/bakztfuture​

Github - https://www.github.com/bakztfuture​

In this episode, we welcome back Abran Maldonado who is an Official OpenAI Community Ambassador.  We discuss all things ChatGPT, OpenAI, and prompt design.

We discuss everything ChatGPT in great detail:

• How it compares to GPT-3
• The public reception of ChatGPT
• Use cases for ChatGPT
• How you can use ChatGPT in schools and as a parent
• Where ChatGPT could be going
• Staying Relevant, Creating, and Adapting
• ... and so much more!

Timestamps:

00:00 - Intro

04:04 - Abran's Background

12:30 - How did you first hear about ChatGPT?

19:46 - ChatGPT First Impressions

28:00 - How do you think about the OpenAI Ecosystem now?

30:38 - Commercializing with the OpenAI API

37:27 - Public Reception - Cool ChatGPT Use cases

40:48 - Public Reception - Jailbreaking ChatGPT

48:40 - Public Reception - ChatGPT vs. Google Search, learning

55:55 - How do you feel about ChatGPT virality?

01:01:25 - Office Hours with OpenAI Ambassadors

01:02:40 - Staying Relevant with ChatGPT

01:11:20 - ChatGPT Browsing the Web (?) WebGPT

01:16:35 - AI and Safety

01:20:30 - The benefit of ChatGPT browsing the web

01:20:09 - ChatGPT Formatting - ChatGPT encylopedia (?)

01:23:43 - ChatGPT and EDU

01:25:08 - Do you have any advice to teachers and parents?

01:30:08 - Language Models and Education Debate, The Future of Education

01:42:40 - Prompt Engineering is dead, long live dialogue engineering!

01:53:57 - Parallels Between Music Scene and the GPT-3/DALL-E/OpenAI Ecosystem

02:00:10 - Music Industry x AI Models

02:05:33 - How to prepare for upcoming Music AI Models?

02:11:05 - Listener Questions

02:20:06 - Closing Thoughts

02:24:00 - Outro

Abran Maldonado 

https://twitter.com/abran

https://www.instagram.com/createlab

Links from this episode:

ChatGPT:

https://openai.com/blog/chatgpt/

Is Prompt Design Over?

https://bakztfuture.substack.com/p/is-prompt-design-over

InstructGPT:

https://openai.com/blog/instruction-following/

InstructGPT vs. google search:

https://bakztfuture.substack.com/p/instructgpt-google-search

WebGPT:

https://openai.com/blog/webgpt/

My Biggest 2022 Prediction: GPT-3 will take over schools and college campuses

https://bakztfuture.substack.com/p/my-biggest-2022-prediction-gpt-3

Subscribe to the Multimodal Podcast!

Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV

Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820

Google Podcasts -  https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s

Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future

Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml

Connect with me:

YouTube - https://www.youtube.com/bakztfuture

Substack Newsletter - https://bakztfuture.substack.com​

Twitter - https://www.twitter.com/bakztfuture​

Instagram - https://www.instagram.com/bakztfuture​

Github - https://www.github.com/bakztfuture​

What could an AI creative tool like DALL-E 2 or Midjourney look like in the next few years?

This video explores the full set of future possibilities upon us!

GPT-X, Diffusion, and our Multimodal Future (Part II) BOOK: https://docs.google.com/document/d/1TeRMqOsGX8kiK_mNiCt4yxO26Yjk6BdM/

Timestamps:

00:00 - Intro
05:02 - Multimodal Model Characteristics
11:26 - Creating a new image
15:56 - Prompt writing capabilities - Prompt Formatting
16:50 - Prompt autocompletion, intelligent suggestions
19:57 - Built in prompt unbundling
21:57 - Configure Recombinants
23:45 - Recombinant blending modes
25:07 - Configure Recombinant Unbundling Details
26:12 - Recombinant Layers & Node based
28:05 - Recombinant Import Bench
29:48 - Natural Language Image editing - refresh individual objects
31:44 - Edit mode - Natural Langauge Edit Prompts
32:37 - Edit prompt realtime feedback 
33:05 - Natural language hex color editing support
34:35 - Advanced Natural Language Edit Prompt Changes
35:53 - Edit prompt intensity (UI controls)
36:59 - Multimodal Object Transformations
37:16 - Lighting & Camera Controls
38:00 - Advanced Editing Tools (Fix AI Weirdness and more)
39:27 - Edit images via markup (markup prompt)
40:43 - Logical Variations
43:57 - Realtime Collaboration
44:28 - Recombinant Collaborators
46:06 - Built in Music Player
48:45 - Creative Hyperparameters
49:20 - Text Capabilities - Magic Text
50:52 - Text Capabilities - Magic Text Fill
52:41 - Latent Variation Scrubbing
53:34 - Explore Alternatives
56:34 - Productivity and commercialization
58:06 - Analysis & Feedback
59:16 - Advanced Multimodal Capabilities
1:03:45 - Offline Capabilities
1:11:11 - Advanced Workflow Support
1:11:58 - Closing thoughts

GPT-X, DALL-E, and our Multimodal Future (Part I):

https://www.youtube.com/playlist?list=PLza3gaByGSXjUCtIuv2x9fwkx3K_3CDmw

Links:

DALL-E 2 Unbundling

https://bakztfuture.substack.com/p/dall-e-2-unbundling

DALL-E 2: Recombinant Art & Design

https://bakztfuture.substack.com/p/dall-e-2-recombinant-art-and-design

DALL-E 2 - Unofficial Natural Language Image Editing, Art Critique Survey

https://bakztfuture.substack.com/p/dall-e-2-unofficial-natural-language-b14

Please note I do not represent, have any affiliation with, nor do I speak on behalf of OpenAI.

Subscribe to the Multimodal Podcast!

Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV

Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820

Google Podcasts -  https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s

Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future

Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml

Connect with me:

YouTube - https://www.youtube.com/bakztfuture

Substack Newsletter - https://bakztfuture.substack.com​

Twitter - https://www.twitter.com/bakztfuture​

Instagram - https://www.instagram.com/bakztfuture​

Github - https://www.github.com/bakztfuture​

September 2022 - State of the Union Slideshow presentation!

Watch the video of this presentation here:

https://www.youtube.com/watch?v=qYvRsLWpMBs

Links from this episode:

GPT-X, DALL-E, and our Multimodal Future:

https://www.youtube.com/playlist?list=PLza3gaByGSXjUCtIuv2x9fwkx3K_3CDmw

InstructGPT:

https://openai.com/blog/instruction-following/

InstructGPT vs. google search:

https://bakztfuture.substack.com/p/instructgpt-google-search

WebGPT:

https://openai.com/blog/webgpt/

Recombinant art:

https://bakztfuture.substack.com/p/dall-e-2-recombinant-art-and-design

DALL-E 2: Emerging Content Category Breakdown:

https://bakztfuture.substack.com/p/dall-e-2-emerging-content-category

DALL-E 2 - Unofficial Natural Language Image Editing, Art Critique Survey

https://bakztfuture.substack.com/p/dall-e-2-unofficial-natural-language-b14

My Biggest 2022 Prediction: GPT-3 will take over schools and college campuses

https://bakztfuture.substack.com/p/my-biggest-2022-prediction-gpt-3

Subscribe to the Multimodal Podcast!

Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV

Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820

Google Podcasts -  https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s

Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future

Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml

Connect with me:

YouTube - https://www.youtube.com/bakztfuture

Substack Newsletter - https://bakztfuture.substack.com​

Twitter - https://www.twitter.com/bakztfuture​

Instagram - https://www.instagram.com/bakztfuture​

Github - https://www.github.com/bakztfuture​

Chatting about Large Language Models with Cohere CTO/Co-founder Nick Frosst

Cohere:

https://cohere.ai/

Nick Frosst:

https://www.nickfrosst.com/

https://twitter.com/nickfrosst

https://scholar.google.ca/citations?user=1yVnaTgAAAAJ

Subscribe to the Multimodal Podcast!

Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV

Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820

Google Podcasts -  https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s

Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future

Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml

Connect with me:

YouTube - https://www.youtube.com/bakztfuture

Substack Newsletter - https://bakztfuture.substack.com​

Twitter - https://www.twitter.com/bakztfuture​

Instagram - https://www.instagram.com/bakztfuture​

Github - https://www.github.com/bakztfuture​