Hands-on with Gemini: Interacting with multimodal AI

Google

Google

6 min, 23 sec

The video involves a male voice testing Gemini, an AI system, with various visual and auditory cues. Gemini responds with detailed descriptions, explanations, and even game and craft ideas.

Summary

  • The male voice tests Gemini's comprehension of visual stimuli by drawing simple pictures.
  • Gemini identifies images such as a squiggly line which it perceives as a bird, a duck, and a blue rubber duck.
  • Gemini explains about the different breeds of ducks, their common colors and the rarity of blue ducks.
  • Gemini plays a game of 'Guess the Country' based on clues given and also participates in a game of 'Rock, Paper, Scissors'.
  • The male voice tests Gemini's ability to understand and respond to auditory cues, like the squeaking of a rubber duck.
  • Gemini is asked to generate craft ideas using colored yarn and to provide the direction a duck in a picture should go based on the symbols present.
  • The male voice also tests Gemini's knowledge of Mandarin, the Chinese language, and how it should be pronounced.
  • The video ends with the male voice asking Gemini to describe a drawing of the constellation Gemini.

Chapter 1

Initial Testing of Gemini

0:17 - 28 sec

The male voice begins by testing Gemini's basic comprehension of visual stimuli.

The male voice begins by testing Gemini's basic comprehension of visual stimuli.

  • The male voice places a piece of paper on the table and asks Gemini to describe what it sees.
  • Gemini describes a squiggly line drawn by the male voice and identifies it as a bird.
  • When more details are added to the drawing, Gemini identifies the bird as a duck.

Chapter 2

Detailed Description of Duck

0:46 - 20 sec

Gemini describes the duck in the picture and gives a detailed explanation about ducks.

Gemini describes the duck in the picture and gives a detailed explanation about ducks.

  • Gemini explains that ducks are a type of waterfowl in the family Anatidae, closely related to geese and swans.
  • Gemini mentions that the duck in the picture appears to be blue, which is not a common color for ducks, but notes the existence of some breeds of blue ducks.

Chapter 3

Further Interaction with Rubber Duck

1:07 - 22 sec

The male voice presents a physical blue rubber duck and Gemini provides predictions based on the duck's characteristics.

The male voice presents a physical blue rubber duck and Gemini provides predictions based on the duck's characteristics.

  • Upon seeing the blue rubber duck, Gemini suggests that blue ducks might be more common than previously stated.
  • When asked if the rubber duck would float, Gemini deduces from the potential material (rubber or plastic) and the squeaking sound, that it would indeed float.

Chapter 4

Language Lessons with Gemini

1:32 - 23 sec

The male voice requests a Mandarin translation from Gemini, and Gemini teaches the pronunciation of the Mandarin word for 'duck'.

The male voice requests a Mandarin translation from Gemini, and Gemini teaches the pronunciation of the Mandarin word for 'duck'.

  • Gemini is asked to translate the word 'duck' into a different language and provide pronunciation tips.
  • Gemini teaches the pronunciation of the Mandarin word for 'duck', 'yāzi', explaining that it is pronounced with a first tone in the Mandarin tonal system.

Chapter 5

Inventive Game Ideas with Gemini

1:55 - 35 sec

The male voice asks Gemini to come up with a game idea based on the images seen, and they play 'Guess the Country'.

The male voice asks Gemini to come up with a game idea based on the images seen, and they play 'Guess the Country'.

  • Gemini suggests a game called 'Guess the Country' based on the images shown.
  • The male voice gives clues about countries, and Gemini attempts to guess the countries based on these clues.

Chapter 6

Further Games and Tests

2:34 - 49 sec

Gemini and the male voice continue playing games and Gemini is tested on various other tasks.

Gemini and the male voice continue playing games and Gemini is tested on various other tasks.

  • Gemini is asked to find a paper ball under a cup, to which it responds correctly.
  • The male voice tests Gemini's comprehension of hand signals used in the game 'Rock, Paper, Scissors'.
  • Gemini correctly identifies a coin hidden under a hand, a test of its observational skills.

Chapter 7

Crafting Ideas with Gemini

3:41 - 27 sec

Gemini is asked to come up with crafting ideas using different colored yarns.

Gemini is asked to come up with crafting ideas using different colored yarns.

  • The male voice presents Gemini with pink and green yarn, and asks for crafting ideas. Gemini suggests making a dragon fruit or a green cake with a pink heart.
  • The male voice then presents blue and pink yarn and asks for crafting ideas with a focus on animals. Gemini suggests a pig with blue ears, an octopus, or a bunny with a pink nose.

Chapter 8

Image Interpretation with Gemini

4:08 - 37 sec

Gemini is asked to interpret various images and scenarios, including identifying a crab and the best direction for a duck to go in a picture.

Gemini is asked to interpret various images and scenarios, including identifying a crab and the best direction for a duck to go in a picture.

  • Gemini interprets an image of a crab correctly.
  • In a picture where a duck has to choose between going towards a friend (another duck) or a foe (a bear), Gemini advises the duck to choose the friend.

Chapter 9

Drawing Interpretation and Guessing Games

4:45 - 1 min, 11 sec

The male voice tests Gemini's ability to interpret drawings and guess possible scenarios.

The male voice tests Gemini's ability to interpret drawings and guess possible scenarios.

  • Gemini is asked to guess what a person in a drawing might be saying and responds with a joyful 'Woohoo!'
  • Gemini correctly identifies a drawing of a guitar and predicts the possible addition of an amplifier to turn it into an electric guitar.
  • Gemini is asked to guess what might happen next in a picture of a cat about to jump and predicts a perfect landing, which unfortunately doesn't happen.

Chapter 10

Final Test: Constellation Drawing

5:56 - 13 sec

The male voice ends the session by asking Gemini to describe a drawing of the constellation Gemini.

The male voice ends the session by asking Gemini to describe a drawing of the constellation Gemini.

  • Gemini correctly identifies the drawing as the constellation Gemini.
  • Gemini compliments the male voice on capturing the beauty of the constellation in his drawing.

More Google summaries

Gemini: Excelling at competitive programming

Gemini: Excelling at competitive programming

Google

Google

Gemini and AlphaCode2 represent advancements in AI-driven programming, with capabilities in understanding, generating, and repairing code, outperforming previous models and human competitors.